-- Logs begin at Tue 2022-06-07 15:05:37 UTC, end at Tue 2022-06-07 16:15:10 UTC. -- Jun 07 15:43:55.705020 dv-cinder-844879-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 15:43:55.716066 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 15:43:55 2022] *** Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: nodename: dv-cinder-844879-2-iscsi Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: machine: x86_64 Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: clock source: unix Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: pcre jit disabled Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: detected number of CPU cores: 4 Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: current working directory: / Jun 07 15:43:55.718114 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: detected binary path: /usr/bin/uwsgi-core Jun 07 15:43:55.718747 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: your processes number limit is 47680 Jun 07 15:43:55.718747 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: your memory page size is 4096 bytes Jun 07 15:43:55.718747 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: detected max file descriptor number: 2048 Jun 07 15:43:55.718747 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: lock engine: pthread robust mutexes Jun 07 15:43:55.718747 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: thunder lock: enabled Jun 07 15:43:55.718747 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 15:43:55.718747 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 15:43:55.762629 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: Python main interpreter initialized at 0x55f87a7559c0 Jun 07 15:43:55.762629 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: python threads support enabled Jun 07 15:43:55.762629 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: your server socket listen backlog is limited to 100 connections Jun 07 15:43:55.762629 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: your mercy for graceful operations on workers is 90 seconds Jun 07 15:43:55.762935 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 15:43:55.763035 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: *** Operational MODE: preforking *** Jun 07 15:43:55.763305 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: *** uWSGI is running in multiple interpreter mode *** Jun 07 15:43:55.763305 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: spawned uWSGI master process (pid: 80309) Jun 07 15:43:55.763483 dv-cinder-844879-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 15:43:55.768430 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: spawned uWSGI worker 1 (pid: 80319, cores: 1) Jun 07 15:43:55.768430 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: spawned uWSGI worker 2 (pid: 80320, cores: 1) Jun 07 15:43:55.768430 dv-cinder-844879-2-iscsi devstack@keystone.service[80309]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 15:44:01.502286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 15:44:01.502286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: from cryptography.utils import int_from_bytes Jun 07 15:44:01.503043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 15:44:01.503043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: from cryptography.utils import int_from_bytes Jun 07 15:44:01.520870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 15:44:01.520870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: from cryptography.utils import int_from_bytes Jun 07 15:44:01.521344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 15:44:01.521344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: from cryptography.utils import int_from_bytes Jun 07 15:44:01.552646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 15:44:01.552841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 15:44:01.553032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] command line args: [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 15:44:01.553299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 15:44:01.553474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ================================================================================ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 15:44:01.553726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] admin_token = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.554001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config_dir = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.554183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.554446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config_source = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.554635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] debug = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.554833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.555340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.555340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.555595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.555839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.556069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.556253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_config_append = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.556455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.556634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_dir = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.556812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.557046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.557290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.557533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.557777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.557960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.558136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.558376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.558618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.558849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.559078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.559311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.559541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.559784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.560028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.560299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.560808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.560989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] publish_errors = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.561219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.561445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.561621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.561862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.562092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.562323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] standard_threads = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.562551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.562790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.562964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.563138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_journal = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.563312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_json = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.563485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_stderr = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.563660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_syslog = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.563885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.564136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.564387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.564649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.564885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.565130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.565363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.565607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.565839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.566082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.566330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.566587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.566820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.567063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.567297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.567540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.567784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.568016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.568266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.568546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.568790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.569024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.569273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.569575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.569827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.570101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.570289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.570516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.570744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.570868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 15:44:01.570982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 15:44:01.571153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.571274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] command line args: [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 15:44:01.571401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.571519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 15:44:01.571637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.571750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ================================================================================ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 15:44:01.571868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.571980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] admin_token = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.572098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.572210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config_dir = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.572319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.572453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.572453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] config_source = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.572670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.572791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] debug = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.572904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.573019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.573160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.573272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.573394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.573511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.573622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.573738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.573857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.573968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.574086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.574197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.574306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.574415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_config_append = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.574415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.574627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.574746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_dir = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.574864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.574982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.575093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.575209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.575325 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.575509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.575622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.575732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.575841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.575951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.576072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.576183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.576294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.576436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.576552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.576663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.576786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.576918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.577080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.577218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.577329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.577440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.577558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.577675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.577786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.577896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.578006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.578121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.578240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.578355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.578472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.578582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.578692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.578692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.578907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.579026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.579138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] publish_errors = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.579248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.579366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.579488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.579600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.579600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.579806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.579923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.580107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.580221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.580331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.580456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] standard_threads = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.580566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.580676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.580798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.580917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.581029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.581138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.581249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.581359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_journal = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.581480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.581592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_json = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.581592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_stderr = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.581806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.581917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] use_syslog = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.582027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.582145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 15:44:01.582258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.582367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.582483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.582483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.582690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.582808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.582924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583371 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583604 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.583941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.584900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.585000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.585635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.585869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.586936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.587946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.588950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.external = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.589977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.password = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.token = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.590991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591438 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.591890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.592925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.593972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.594977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.595914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.596890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.597910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.598021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.598140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.598253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.598364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.598475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.598585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.598958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.599905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.600491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.601857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.601857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.602980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.603896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.604967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.605984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.606958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.external = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.607978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.password = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] auth.token = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.608911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.609959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.610881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.611912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.612940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.613965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.614976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.615998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.616109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.616220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.616402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.616537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.616659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.616780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.616895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.617909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.618022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.618022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.618232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.618344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.618566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.618812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.619965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.620912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.621945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.622963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.623985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.624944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.625965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.626980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627350 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.627927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628406 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.628980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.629977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.630998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.631109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.631220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.631331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.631450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.631564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.631676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.631786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.632444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.632562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.632686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.632913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.633922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.634948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.635926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636167 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636757 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.636979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637325 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.637891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.638901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639694 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.639927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.640038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.640149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.640260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 15:44:01.640427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.640543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.640717 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.640958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.641199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.641437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.641685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.641915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.642165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.642412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.642660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.642908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.643155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.643401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.643648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.643895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.644130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.644412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.644642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.644890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.645138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.645545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.645785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.646038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.646288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.646538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.646788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.647024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.647279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.647590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.647912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.648252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.648593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.648834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.649091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.649349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.649600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.649847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.650086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.650318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.650556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.650794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.651044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.651282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.651539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.651788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.652026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.652271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.652539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.652789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.653027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.653266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.653502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.653749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.653995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.654225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.654466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.654766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.655027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.655275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.655522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.655762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.656003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.656238 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.656502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.656738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.656975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.657220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.657455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.657700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.657938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.658186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.658424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.658671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.658903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.659143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.659379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 15:44:01.659550 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 15:44:01.669173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.669405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.669629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.677941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.685083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.685315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.685539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.693852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.706193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.706424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.712577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.712809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.760527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.760963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.809949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.810174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.820787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.821179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.893194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.893424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.893650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.893874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.894093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.894307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.895411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.895645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.895872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.896096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.896316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.896578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.917721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.917973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.918902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.919029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.919109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.919398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.919622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.930545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.943386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.943618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.943767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.943895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.944965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.945255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80319) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 15:44:01.962454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.962553 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.962667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 15:44:01.962750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 15:44:01.963004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.963087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.963205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 15:44:01.963287 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 15:44:01.963467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.963563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.963722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.963804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.963976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.964057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.964226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.964312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.975274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.975367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.975549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.976124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.976462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.976561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.976749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.976832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.985038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:01.985129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:01.988305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:01.988536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:01.988619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.988819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.988953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.989980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.990162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.990242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.990425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.990512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.990690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.990769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.990950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.991028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:01.991212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.017944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.018284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.019507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.019786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.019908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.020187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.025832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.026110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.026273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.026386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.026551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.026828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.033681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.033963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.034225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.034511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.038769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.039076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.039352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.039483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.039643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.039772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.039914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.040056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.040182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.040339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.040474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.040641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.040853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.040933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.041169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.041436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.059889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.060161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.060774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.061051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.064795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.065636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.068547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.068840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.069485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.069778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.073911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.074823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.077527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.078431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.081151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.081448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.081730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.082010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.082117 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.082298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.082457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.082582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.082664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.082951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.083226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.083511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.102773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.103050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.103313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.103554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.103693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.103844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.104116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.104405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.111470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.112279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.114703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.115003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.115440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.115729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.118939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.119220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.119453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.119536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.119735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.119816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.120014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.120096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.120298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.120597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.133543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.133833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.134112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.134202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.134392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.134607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.134698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.134941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.135022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.135226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.135428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.135853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.136255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.136699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.155508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.158822 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.160523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.162306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.164610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.165496 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.166265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.166545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.166820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.168998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.170628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.172257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.172561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.172979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.174143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.175757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.179201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.179483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.179762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.180036 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.185385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.188116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.189119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.189405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.189682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.192725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.193017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.193358 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.193645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.197694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.197971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.198244 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.198514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.198783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.199050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.199319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.199588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.199866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.200140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.200431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.200699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.200969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.201224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.206346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.206639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.207057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.218675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80319) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 15:44:02.219090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.219492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.219890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.220279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.220694 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.221091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.221494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.221892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.222284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.222690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.223077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.223468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.224123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80319) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 15:44:02.240170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f87a7559c0 pid: 80320 (default app) Jun 07 15:44:02.243694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435624b-979c-4de5-8e91-c788a1932a35 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:02.243979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435624b-979c-4de5-8e91-c788a1932a35 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:02.244254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435624b-979c-4de5-8e91-c788a1932a35 None None] PATH_INFO: `/v3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:02.248428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Tue Jun 7 15:44:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:02.267884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f87a7559c0 pid: 80319 (default app) Jun 07 15:44:10.258147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c58b10-700c-403b-9107-eff7bf6a6c26 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:10.258693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c58b10-700c-403b-9107-eff7bf6a6c26 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:10.258693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c58b10-700c-403b-9107-eff7bf6a6c26 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:10.260029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 2/2] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:10.267762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:10.268150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:10.268321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:10.337481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: INFO dbcounter [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] Registered counter for database keystone Jun 07 15:44:10.359057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG oslo_db.sqlalchemy.engines [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80319) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 15:44:10.365355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writer thread running {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 15:44:10.645434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG passlib.handlers.bcrypt [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80319) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 15:44:10.645864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG passlib.handlers.bcrypt [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80319) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 15:44:10.701800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:10.703405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:10.827030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1/3] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 563 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 07 15:44:10.986088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c58b10-700c-403b-9107-eff7bf6a6c26 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:11.054949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: INFO dbcounter [None req-26c58b10-700c-403b-9107-eff7bf6a6c26 None None] Registered counter for database keystone Jun 07 15:44:11.074565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG oslo_db.sqlalchemy.engines [None req-26c58b10-700c-403b-9107-eff7bf6a6c26 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80320) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 15:44:11.080688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writer thread running {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 15:44:11.315137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c58b10-700c-403b-9107-eff7bf6a6c26 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:11.319895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39867894-829d-4e01-aeb9-e437e8311ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:11.320576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867894-829d-4e01-aeb9-e437e8311ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:11.320864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867894-829d-4e01-aeb9-e437e8311ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:11.321137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867894-829d-4e01-aeb9-e437e8311ee2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:11.329204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39867894-829d-4e01-aeb9-e437e8311ee2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:11.382500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING keystone.common.rbac_enforcer.enforcer [None req-39867894-829d-4e01-aeb9-e437e8311ee2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 15:44:11.382840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39867894-829d-4e01-aeb9-e437e8311ee2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:11.392185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 3/4] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Jun 7 15:44:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 15:44:11.398295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:11.412557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6de746-7ecb-42e5-9e37-078bf3090f4c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:11.417664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b46eaa-5dfc-45bc-9e4a-6444e8f2d8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:11.418783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46eaa-5dfc-45bc-9e4a-6444e8f2d8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:11.419070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46eaa-5dfc-45bc-9e4a-6444e8f2d8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:11.419344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46eaa-5dfc-45bc-9e4a-6444e8f2d8c1 admin admin] PATH_INFO: `/v3/projects/3e30cc2a9bc04d579b04c2060b966f87` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:11.420857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b46eaa-5dfc-45bc-9e4a-6444e8f2d8c1 admin admin] RBAC: Authorizing `identity:get_project(project_id=3e30cc2a9bc04d579b04c2060b966f87)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:11.472845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING keystone.common.rbac_enforcer.enforcer [None req-99b46eaa-5dfc-45bc-9e4a-6444e8f2d8c1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 15:44:11.473183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b46eaa-5dfc-45bc-9e4a-6444e8f2d8c1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:11.477934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 2/5] 10.222.0.18 () {56 vars in 1207 bytes} [Tue Jun 7 15:44:11 2022] GET /identity/v3/projects/3e30cc2a9bc04d579b04c2060b966f87 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:13.705229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30672e7-9b67-48bc-8ade-06d5306bd6ca None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:13.705754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30672e7-9b67-48bc-8ade-06d5306bd6ca None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:13.705850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30672e7-9b67-48bc-8ade-06d5306bd6ca None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:13.710048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 4/6] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:13.715531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13edc0-8719-480e-9c7f-7edc7d818d00 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:13.715914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13edc0-8719-480e-9c7f-7edc7d818d00 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:13.716088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13edc0-8719-480e-9c7f-7edc7d818d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:13.736701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cf13edc0-8719-480e-9c7f-7edc7d818d00 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:13.738205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cf13edc0-8719-480e-9c7f-7edc7d818d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:13.746722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 3/7] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 15:44:13.769095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30672e7-9b67-48bc-8ade-06d5306bd6ca None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:13.781597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30672e7-9b67-48bc-8ade-06d5306bd6ca None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:13.785793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4405948-b5af-4a97-878d-012d6bc7eab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:13.786426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405948-b5af-4a97-878d-012d6bc7eab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:13.786714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405948-b5af-4a97-878d-012d6bc7eab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:13.786993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405948-b5af-4a97-878d-012d6bc7eab2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:13.795353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4405948-b5af-4a97-878d-012d6bc7eab2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:13.797170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4405948-b5af-4a97-878d-012d6bc7eab2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:13.805993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 5/8] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Jun 7 15:44:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 15:44:13.811541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13edc0-8719-480e-9c7f-7edc7d818d00 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:13.821013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13edc0-8719-480e-9c7f-7edc7d818d00 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:13.825610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e11a8a-34da-4060-8e86-11539ec98675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:13.826473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e11a8a-34da-4060-8e86-11539ec98675 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:13.826759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e11a8a-34da-4060-8e86-11539ec98675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:13.827035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e11a8a-34da-4060-8e86-11539ec98675 admin admin] PATH_INFO: `/v3/users/bd49b97115e3460492249a2e84a91b53` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:13.828462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e11a8a-34da-4060-8e86-11539ec98675 admin admin] RBAC: Authorizing `identity:get_user(user_id=bd49b97115e3460492249a2e84a91b53)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:13.830290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e11a8a-34da-4060-8e86-11539ec98675 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:13.831863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 4/9] 10.222.0.18 () {56 vars in 1198 bytes} [Tue Jun 7 15:44:13 2022] GET /identity/v3/users/bd49b97115e3460492249a2e84a91b53 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:16.235133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc438b0-06b8-4d70-b7df-c27f91e3a45f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.235666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc438b0-06b8-4d70-b7df-c27f91e3a45f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.235666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc438b0-06b8-4d70-b7df-c27f91e3a45f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.237059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 6/10] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:16.243702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6b9d-c5f8-4bb1-97f6-86a812f56999 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.244205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6b9d-c5f8-4bb1-97f6-86a812f56999 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.244334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6b9d-c5f8-4bb1-97f6-86a812f56999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.261722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-357a6b9d-c5f8-4bb1-97f6-86a812f56999 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:16.262207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b47a4-4ef3-486e-adae-9a6533e951de None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.262712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b47a4-4ef3-486e-adae-9a6533e951de None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.262908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-357a6b9d-c5f8-4bb1-97f6-86a812f56999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:16.263208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b47a4-4ef3-486e-adae-9a6533e951de None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.265211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 7/11] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:16.270360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.270657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.270940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.312296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG passlib.handlers.bcrypt [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80320) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 15:44:16.312746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG passlib.handlers.bcrypt [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80320) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 15:44:16.346703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 5/12] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:16.360045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:16.361681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:16.369533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6b9d-c5f8-4bb1-97f6-86a812f56999 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:16.394205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:16.463493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 8/13] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:16.479797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6b9d-c5f8-4bb1-97f6-86a812f56999 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:16.487472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:16.512823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:16.513477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.513760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.514047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.580486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:16.581965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:16.585860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ff31-c286-476b-b207-acdbe7ea4b7d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:16.588143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:16.605840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:16.623420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:16.624568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.624926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.625294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.638267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:16.650134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:16.651261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:16.652147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 9/14] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:16.657072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:16.686405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 6/15] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 15:44:16.691967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:16.757688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626568ab-e2f6-4f6e-8431-955630b0f7a1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:16.789207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ea77f-1b1e-41eb-8bf1-ebf1c0b91fc8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:16.801516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:16.802184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.802462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.802739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.804016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:16.804310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:16.805788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:16.812949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:16.817546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 10/16] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:16.822829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:16.825413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:16.826209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.826485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.826755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.836785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:16.838235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:16.849794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:44:16.859355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:16.860242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:16.861044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:44:16.861044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:44:16.861044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:44:16.861044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:16.861044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 7/17] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:16.866942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:16.932569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf1ef3-6515-4d29-ba29-35b279ea1c64 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:16.966120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9725339-7f02-4c44-82e1-a8d24556d0ab None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:16.973539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:16.974679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:16.974965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:16.975805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] PATH_INFO: `/v3/users/bd49b97115e3460492249a2e84a91b53` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:16.977516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] RBAC: Authorizing `identity:get_user(user_id=bd49b97115e3460492249a2e84a91b53)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:16.979562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:16.982347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 11/18] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:16 2022] GET /identity/v3/users/bd49b97115e3460492249a2e84a91b53 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:16.987484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:17.000412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:17.001086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:17.001371 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:17.001650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:17.002381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:17.002691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:17.004085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:17.021049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 8/19] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:17.087583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:17.095666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7494bf-5475-4783-927b-ea0752cc93a5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:17.134628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b0154-1641-45f2-924f-df479cb1d839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:17.135449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0154-1641-45f2-924f-df479cb1d839 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:17.135732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0154-1641-45f2-924f-df479cb1d839 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:17.136007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0154-1641-45f2-924f-df479cb1d839 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:17.137267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0154-1641-45f2-924f-df479cb1d839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:17.138754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0154-1641-45f2-924f-df479cb1d839 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:17.140586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 12/20] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:17.146311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:17.183683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:17.258127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455033b-3de2-482b-854a-ce25028c846d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:17.302006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9217-af87-4a48-b568-47f42798acad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:17.302606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9217-af87-4a48-b568-47f42798acad None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:17.302887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9217-af87-4a48-b568-47f42798acad None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:17.303162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9217-af87-4a48-b568-47f42798acad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:17.304002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f9217-af87-4a48-b568-47f42798acad None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.domain.id=default, user.id=bd49b97115e3460492249a2e84a91b53) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:17.304308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f9217-af87-4a48-b568-47f42798acad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:17.305958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f9217-af87-4a48-b568-47f42798acad None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:17.313362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 9/21] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Jun 7 15:44:17 2022] GET /identity/v3/role_assignments?user.id=bd49b97115e3460492249a2e84a91b53&scope.domain.id=default&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 233 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:19.866152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c1d2-0145-425a-a114-63704d1189ce None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:19.867162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c1d2-0145-425a-a114-63704d1189ce None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:19.867588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c1d2-0145-425a-a114-63704d1189ce None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:19.869453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 13/22] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:19.875473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc7375-877c-4ca9-b9f4-e035055881ec None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:19.876231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc7375-877c-4ca9-b9f4-e035055881ec None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:19.876692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc7375-877c-4ca9-b9f4-e035055881ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:19.905426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-52dc7375-877c-4ca9-b9f4-e035055881ec None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:19.906649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-52dc7375-877c-4ca9-b9f4-e035055881ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:19.993479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 10/23] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:20.032617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74c1d2-0145-425a-a114-63704d1189ce None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:20.034823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad133-ac63-4700-8e42-0e37708ea6a0 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.035121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad133-ac63-4700-8e42-0e37708ea6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.035411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad133-ac63-4700-8e42-0e37708ea6a0 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.037029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 11/24] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:20.044387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83530a1-29ab-47b5-b1eb-9909d278cadb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.044387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83530a1-29ab-47b5-b1eb-9909d278cadb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.044387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83530a1-29ab-47b5-b1eb-9909d278cadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.071717 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d83530a1-29ab-47b5-b1eb-9909d278cadb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:20.072968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d83530a1-29ab-47b5-b1eb-9909d278cadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:20.134661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:20.222953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 12/25] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:20.270377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83530a1-29ab-47b5-b1eb-9909d278cadb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:20.270810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74c1d2-0145-425a-a114-63704d1189ce None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:20.308157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a5067-3530-4ac5-9c73-1bd9741cd16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:20.308993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a5067-3530-4ac5-9c73-1bd9741cd16c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.309275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a5067-3530-4ac5-9c73-1bd9741cd16c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.309555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a5067-3530-4ac5-9c73-1bd9741cd16c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.329377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a5067-3530-4ac5-9c73-1bd9741cd16c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:20.330882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a5067-3530-4ac5-9c73-1bd9741cd16c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b07a5067-3530-4ac5-9c73-1bd9741cd16c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:44:20.351517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:20.352417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:20.353194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:44:20.353194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:44:20.353194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:44:20.353194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:20.353194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 14/26] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:20.354155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641ddea-1934-4a99-b34c-0132aefa0539 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.354433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641ddea-1934-4a99-b34c-0132aefa0539 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.354721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641ddea-1934-4a99-b34c-0132aefa0539 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.356049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 15/27] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:20.361927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70792670-c069-45e2-a2f3-7e5d4c875edb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.362274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70792670-c069-45e2-a2f3-7e5d4c875edb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.362557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70792670-c069-45e2-a2f3-7e5d4c875edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.403108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-70792670-c069-45e2-a2f3-7e5d4c875edb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:20.404401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-70792670-c069-45e2-a2f3-7e5d4c875edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:20.415218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:20.476761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 16/28] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:20.477272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70792670-c069-45e2-a2f3-7e5d4c875edb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:20.486236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83530a1-29ab-47b5-b1eb-9909d278cadb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:20.550427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5439438-b147-4500-bb9b-62e45bb3a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:20.551115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5439438-b147-4500-bb9b-62e45bb3a746 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.551401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5439438-b147-4500-bb9b-62e45bb3a746 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.551679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5439438-b147-4500-bb9b-62e45bb3a746 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.552461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5439438-b147-4500-bb9b-62e45bb3a746 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:20.554080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5439438-b147-4500-bb9b-62e45bb3a746 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:20.587450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70792670-c069-45e2-a2f3-7e5d4c875edb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:20.627177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:20.627863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.628271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.628685 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.629516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:20.629877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:20.631660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:20.651949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 17/29] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:20.652900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:20.656925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:20.722691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 13/30] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Jun 7 15:44:20 2022] POST /identity/v3/roles => generated 219 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:44:20.724662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44041024-70c0-4f27-b810-0fe2ab2be2e7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.724950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44041024-70c0-4f27-b810-0fe2ab2be2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.726266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44041024-70c0-4f27-b810-0fe2ab2be2e7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.732462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 14/31] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:20.733440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44041024-70c0-4f27-b810-0fe2ab2be2e7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:20.767352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12f10b-2896-4aed-849e-bef5dfd4461f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:20.797169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:20.814674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55065f-5aa6-4ee1-9bf5-26c320f1816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:20.815369 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55065f-5aa6-4ee1-9bf5-26c320f1816c None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.815653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55065f-5aa6-4ee1-9bf5-26c320f1816c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.815930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55065f-5aa6-4ee1-9bf5-26c320f1816c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.816738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55065f-5aa6-4ee1-9bf5-26c320f1816c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:20.818985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55065f-5aa6-4ee1-9bf5-26c320f1816c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:20.935159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44041024-70c0-4f27-b810-0fe2ab2be2e7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:20.979517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d400aef-706b-4b5d-b60a-9bd5b8c43000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:20.980189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d400aef-706b-4b5d-b60a-9bd5b8c43000 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.980510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d400aef-706b-4b5d-b60a-9bd5b8c43000 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.980790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d400aef-706b-4b5d-b60a-9bd5b8c43000 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:20.981511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d400aef-706b-4b5d-b60a-9bd5b8c43000 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:20.981985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 18/32] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Jun 7 15:44:20 2022] POST /identity/v3/roles => generated 225 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:20.983157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d400aef-706b-4b5d-b60a-9bd5b8c43000 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:20.983508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845b92-ce78-453a-a377-bae95e0b5ed5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:20.983816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845b92-ce78-453a-a377-bae95e0b5ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:20.984105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845b92-ce78-453a-a377-bae95e0b5ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.005201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d4845b92-ce78-453a-a377-bae95e0b5ed5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:21.007358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d4845b92-ce78-453a-a377-bae95e0b5ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:21.027777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 15/33] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:44:20 2022] POST /identity/v3/projects => generated 301 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:21.029397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d0d7-461f-41ea-8080-3a7c264087cc None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.029799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d0d7-461f-41ea-8080-3a7c264087cc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.030185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d0d7-461f-41ea-8080-3a7c264087cc None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.031643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 16/34] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:21.038185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fd560-5fb6-4e86-ab7a-8c09a9d7fdcf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.038624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fd560-5fb6-4e86-ab7a-8c09a9d7fdcf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.039015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fd560-5fb6-4e86-ab7a-8c09a9d7fdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.071048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-208fd560-5fb6-4e86-ab7a-8c09a9d7fdcf None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:21.072783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-208fd560-5fb6-4e86-ab7a-8c09a9d7fdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:21.149553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:21.174728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 19/35] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:21.193615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845b92-ce78-453a-a377-bae95e0b5ed5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.208394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:21.218764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 17/36] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:21.277506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fd560-5fb6-4e86-ab7a-8c09a9d7fdcf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.354318 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845b92-ce78-453a-a377-bae95e0b5ed5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:21.410207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:21.410753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.411035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.411310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.423565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:21.430277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fd560-5fb6-4e86-ab7a-8c09a9d7fdcf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:21.432129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:21.433048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:21.433803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 20/37] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:21.440170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.471286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65092384-08cc-4247-abd0-320dc64dda4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:21.472098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65092384-08cc-4247-abd0-320dc64dda4f None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.472547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65092384-08cc-4247-abd0-320dc64dda4f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.472830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65092384-08cc-4247-abd0-320dc64dda4f None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.473594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65092384-08cc-4247-abd0-320dc64dda4f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:21.475270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65092384-08cc-4247-abd0-320dc64dda4f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:21.491621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 18/38] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Jun 7 15:44:21 2022] POST /identity/v3/roles => generated 223 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:21.503905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:21.557436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c276d4-d8e1-4498-8f4b-315b9f5a5206 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:21.581116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eb4a4-2f9a-457b-b13d-6bb315a55271 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.581437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eb4a4-2f9a-457b-b13d-6bb315a55271 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.581704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eb4a4-2f9a-457b-b13d-6bb315a55271 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.583245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 19/39] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:21.590585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeba42-0449-48ce-8971-80befcadc2ef None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.590868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeba42-0449-48ce-8971-80befcadc2ef None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.591154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeba42-0449-48ce-8971-80befcadc2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.597850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:21.598538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.599073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.599346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.600848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:21.601260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:21.602851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:21.610039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3bdeba42-0449-48ce-8971-80befcadc2ef None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:21.611285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3bdeba42-0449-48ce-8971-80befcadc2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:21.612519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 21/40] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:21.618168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.677619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:21.695755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 20/41] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:21.718426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeba42-0449-48ce-8971-80befcadc2ef None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.736548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834735b-e67a-424a-95cf-680f3a37fd26 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:21.754981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:21.776529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:21.777408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.777691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.777967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] PATH_INFO: `/v3/users/bd49b97115e3460492249a2e84a91b53` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.779144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] RBAC: Authorizing `identity:get_user(user_id=bd49b97115e3460492249a2e84a91b53)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:21.780619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:21.782441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 22/42] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:21 2022] GET /identity/v3/users/bd49b97115e3460492249a2e84a91b53 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:21.787446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.825189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeba42-0449-48ce-8971-80befcadc2ef None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:21.859372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:21.860154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.860470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.860750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.861930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:21.863384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:21.865024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 21/43] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:21.870908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.887917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c516-d71b-4ef3-8759-c21541bb68b0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:21.920868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:21.921637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:21.921923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:21.922201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:21.923359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:21.924837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:21.926429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 23/44] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:21.927117 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:21.932740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:21.969483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71551ec6-0819-46ab-9120-8b0789aa8962 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:21.989090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:22.008291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5077c1-f5d2-4766-bff3-1e80821875c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:22.008984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5077c1-f5d2-4766-bff3-1e80821875c3 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:22.009270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5077c1-f5d2-4766-bff3-1e80821875c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:22.009545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5077c1-f5d2-4766-bff3-1e80821875c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:22.010255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5077c1-f5d2-4766-bff3-1e80821875c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:22.011698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5077c1-f5d2-4766-bff3-1e80821875c3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:22.037180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 22/45] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:44:21 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:44:22.046119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e774-0e51-4174-911a-519bc0b23c78 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:22.083570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d61670-dc03-4a6b-b4c8-cf1d19b94c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:22.084608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d61670-dc03-4a6b-b4c8-cf1d19b94c1c None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:22.085055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d61670-dc03-4a6b-b4c8-cf1d19b94c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:22.085550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d61670-dc03-4a6b-b4c8-cf1d19b94c1c None admin] PATH_INFO: `/v3/domains/default/users/bd49b97115e3460492249a2e84a91b53/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:22.087767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d61670-dc03-4a6b-b4c8-cf1d19b94c1c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bd49b97115e3460492249a2e84a91b53, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:22.089264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d61670-dc03-4a6b-b4c8-cf1d19b94c1c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:22.105981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 24/46] 10.222.0.18 () {58 vars in 1361 bytes} [Tue Jun 7 15:44:21 2022] PUT /identity/v3/domains/default/users/bd49b97115e3460492249a2e84a91b53/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:44:24.350935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15894a6-43dd-433b-859a-2a3e0ed1a15a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.351487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15894a6-43dd-433b-859a-2a3e0ed1a15a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.351713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15894a6-43dd-433b-859a-2a3e0ed1a15a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.353329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:24.358861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc92d9-f468-474c-8e1b-07d6e2973503 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.359304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc92d9-f468-474c-8e1b-07d6e2973503 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.359418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc92d9-f468-474c-8e1b-07d6e2973503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.361955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d79829-4d9d-4158-a560-f5852d70bf6a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.362248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d79829-4d9d-4158-a560-f5852d70bf6a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.362531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d79829-4d9d-4158-a560-f5852d70bf6a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.363820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 24/48] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:24.369226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc7f3-77bc-4c86-899a-4a807e81f22a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.369512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc7f3-77bc-4c86-899a-4a807e81f22a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.369790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc7f3-77bc-4c86-899a-4a807e81f22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.376981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4afc92d9-f468-474c-8e1b-07d6e2973503 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:24.378160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4afc92d9-f468-474c-8e1b-07d6e2973503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:24.386310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-89afc7f3-77bc-4c86-899a-4a807e81f22a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:24.387512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-89afc7f3-77bc-4c86-899a-4a807e81f22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:24.445238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 25/49] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:24.453458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 25/50] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:24.468816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc92d9-f468-474c-8e1b-07d6e2973503 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:24.477458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc7f3-77bc-4c86-899a-4a807e81f22a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:24.511026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:24.518428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:24.570605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc92d9-f468-474c-8e1b-07d6e2973503 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:24.577453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc7f3-77bc-4c86-899a-4a807e81f22a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:24.604678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:24.605508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.605784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.606056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.607219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:24.608711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:24.610305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 26/51] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:24.611114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:24.611911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.612196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.612515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.616295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:24.624499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:24.633985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:24.634811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:24.635547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:24.636401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 26/52] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:24.640774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:24.712713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380dcad-39cf-4960-b31a-34b1f27084aa None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:24.723118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:24.738417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:24.742063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb971b-d312-45a5-821a-b9c5931d6ee3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:24.747691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:24.748370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.748671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.748950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.749662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:24.751099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:24.776802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:24.777478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.777771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.778051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.778318 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 27/53] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Jun 7 15:44:24 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:44:24.778805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:24.779116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:24.780516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:24.797130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 27/54] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:24.802995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:24.911763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfb66b-ba18-450b-a1de-6afc45cbb143 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:24.948329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:24.949257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:24.949542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:24.949821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] PATH_INFO: `/v3/users/bd49b97115e3460492249a2e84a91b53` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:24.951071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] RBAC: Authorizing `identity:get_user(user_id=bd49b97115e3460492249a2e84a91b53)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:24.952570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:24.954260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 28/55] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:24 2022] GET /identity/v3/users/bd49b97115e3460492249a2e84a91b53 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:24.960069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:25.062573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442179f3-10b3-48f3-ae73-739b6283e866 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:25.095642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a706361-53d3-44cd-9e58-a43afcd848db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:25.096464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706361-53d3-44cd-9e58-a43afcd848db None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:25.096755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706361-53d3-44cd-9e58-a43afcd848db None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:25.097036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706361-53d3-44cd-9e58-a43afcd848db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:25.098211 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a706361-53d3-44cd-9e58-a43afcd848db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:25.100462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a706361-53d3-44cd-9e58-a43afcd848db None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:25.102177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 28/56] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:25.107803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:25.140208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:25.211379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397a2e-f156-44c5-a62b-dad97b1e391d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:25.249052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8056b4-2659-4f4d-a86c-7ba68b47857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:25.249746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8056b4-2659-4f4d-a86c-7ba68b47857b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:25.250014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8056b4-2659-4f4d-a86c-7ba68b47857b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:25.250292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8056b4-2659-4f4d-a86c-7ba68b47857b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:25.251130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8056b4-2659-4f4d-a86c-7ba68b47857b None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.domain.id=default, user.id=bd49b97115e3460492249a2e84a91b53) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:25.251447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8056b4-2659-4f4d-a86c-7ba68b47857b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:25.252943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8056b4-2659-4f4d-a86c-7ba68b47857b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:25.261687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 29/57] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Jun 7 15:44:25 2022] GET /identity/v3/role_assignments?user.id=bd49b97115e3460492249a2e84a91b53&scope.domain.id=default&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:27.026639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9bd9a-8f85-41e2-90a1-4fdfc8246eef None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:27.027159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9bd9a-8f85-41e2-90a1-4fdfc8246eef None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:27.027265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9bd9a-8f85-41e2-90a1-4fdfc8246eef None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:27.028594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 29/58] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:27.033622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed28b-ce9a-48c3-b1b9-2382ac1d2b94 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:27.034032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed28b-ce9a-48c3-b1b9-2382ac1d2b94 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:27.034185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed28b-ce9a-48c3-b1b9-2382ac1d2b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:27.051281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6d7ed28b-ce9a-48c3-b1b9-2382ac1d2b94 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:27.052471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6d7ed28b-ce9a-48c3-b1b9-2382ac1d2b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:27.119444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 30/59] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:27.143427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9bd9a-8f85-41e2-90a1-4fdfc8246eef None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:27.173197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:27.242963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9bd9a-8f85-41e2-90a1-4fdfc8246eef None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:27.277232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585791c-4b78-4974-9bd4-a2a6b22c787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:27.277947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585791c-4b78-4974-9bd4-a2a6b22c787c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:27.278228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585791c-4b78-4974-9bd4-a2a6b22c787c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:27.278502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585791c-4b78-4974-9bd4-a2a6b22c787c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:27.279674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585791c-4b78-4974-9bd4-a2a6b22c787c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:27.281138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585791c-4b78-4974-9bd4-a2a6b22c787c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:27.282732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 30/60] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:27.288670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ed28b-ce9a-48c3-b1b9-2382ac1d2b94 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:27.313048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:27.386575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ed28b-ce9a-48c3-b1b9-2382ac1d2b94 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:27.420375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fbc94-0cd2-4cd3-8d42-06dd419a2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:27.421053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fbc94-0cd2-4cd3-8d42-06dd419a2845 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:27.421335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fbc94-0cd2-4cd3-8d42-06dd419a2845 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:27.421611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fbc94-0cd2-4cd3-8d42-06dd419a2845 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:27.422324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fbc94-0cd2-4cd3-8d42-06dd419a2845 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:27.423772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fbc94-0cd2-4cd3-8d42-06dd419a2845 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:27.448809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 31/61] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:44:27 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:44:30.914333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c395-4e33-485b-81f9-6217cc52c50b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:30.915382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c395-4e33-485b-81f9-6217cc52c50b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:30.915865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c395-4e33-485b-81f9-6217cc52c50b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:30.917481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 31/62] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:30.923093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85747-d5bb-40f0-a1f4-d32cb210cc63 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:30.923958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85747-d5bb-40f0-a1f4-d32cb210cc63 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:30.924487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85747-d5bb-40f0-a1f4-d32cb210cc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:30.946452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e925-73b9-4505-b0a3-81bac018ab42 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:30.947056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e925-73b9-4505-b0a3-81bac018ab42 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:30.947526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e925-73b9-4505-b0a3-81bac018ab42 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:30.949159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 32/63] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:30.954336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-edc85747-d5bb-40f0-a1f4-d32cb210cc63 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:30.955769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-edc85747-d5bb-40f0-a1f4-d32cb210cc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:30.963515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a10ac-1d87-4160-a4a3-28dcf704a9fd None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:30.964083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a10ac-1d87-4160-a4a3-28dcf704a9fd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:30.964621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a10ac-1d87-4160-a4a3-28dcf704a9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:30.988712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-896a10ac-1d87-4160-a4a3-28dcf704a9fd None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:30.991126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-896a10ac-1d87-4160-a4a3-28dcf704a9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:31.077476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 32/64] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:31.078983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e552040-4c58-47db-b5f3-ba4187971bbd None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.079284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e552040-4c58-47db-b5f3-ba4187971bbd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.079677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e552040-4c58-47db-b5f3-ba4187971bbd None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.081106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 33/65] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:31.086769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49458180-53fd-40f8-928d-e2cf5f65b4f5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.087064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49458180-53fd-40f8-928d-e2cf5f65b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.087347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49458180-53fd-40f8-928d-e2cf5f65b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.092421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 33/66] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:31.105674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-49458180-53fd-40f8-928d-e2cf5f65b4f5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:31.106869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-49458180-53fd-40f8-928d-e2cf5f65b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:31.109879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.118135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a10ac-1d87-4160-a4a3-28dcf704a9fd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.147464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.185568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 34/67] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:31.194114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49458180-53fd-40f8-928d-e2cf5f65b4f5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.225877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a10ac-1d87-4160-a4a3-28dcf704a9fd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.261833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c8488-492d-4df7-86e2-d0df3952a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.262634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c8488-492d-4df7-86e2-d0df3952a6ec None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.262920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c8488-492d-4df7-86e2-d0df3952a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.263206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c8488-492d-4df7-86e2-d0df3952a6ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.274779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c8488-492d-4df7-86e2-d0df3952a6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-682c8488-492d-4df7-86e2-d0df3952a6ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:31.284920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:31.285772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:31.286546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:31.291857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 34/68] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:31.292873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aa7d2-3c89-4ae3-b3ed-15dcc3e78c1b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.293175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aa7d2-3c89-4ae3-b3ed-15dcc3e78c1b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.293460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aa7d2-3c89-4ae3-b3ed-15dcc3e78c1b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.294807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 35/69] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:31.295499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0aa7d2-3c89-4ae3-b3ed-15dcc3e78c1b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.332110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49458180-53fd-40f8-928d-e2cf5f65b4f5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.370023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.371059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.371486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.371932 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.380307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.383500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:31.392342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:31.393231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:31.393972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 35/70] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:31.394584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.403620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.418863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0aa7d2-3c89-4ae3-b3ed-15dcc3e78c1b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.455139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3d6b9-3dba-443f-a16d-5f3162cea657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.455933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3d6b9-3dba-443f-a16d-5f3162cea657 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.456228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3d6b9-3dba-443f-a16d-5f3162cea657 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.456540 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3d6b9-3dba-443f-a16d-5f3162cea657 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.467946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3d6b9-3dba-443f-a16d-5f3162cea657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2cf3d6b9-3dba-443f-a16d-5f3162cea657 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:31.476438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:31.477285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:31.478066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 36/71] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:31.478859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1285-c94a-49cc-b5a7-4dbdd3cf6c5c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.479295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1285-c94a-49cc-b5a7-4dbdd3cf6c5c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.479722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1285-c94a-49cc-b5a7-4dbdd3cf6c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.497533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-611f1285-c94a-49cc-b5a7-4dbdd3cf6c5c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:31.498736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-611f1285-c94a-49cc-b5a7-4dbdd3cf6c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:31.509472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4a8b6-3e9f-4f88-a962-f9d92c8fe738 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.546216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.546897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.547186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.547477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.548228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:31.548569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:31.550207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:31.563859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 36/72] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:31.563859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.573758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 37/73] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:31.574600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1285-c94a-49cc-b5a7-4dbdd3cf6c5c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.637961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.659546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.679409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92a01a-6fa0-46b5-a616-bcb47390cf40 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.681879 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1285-c94a-49cc-b5a7-4dbdd3cf6c5c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.712004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.712649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.712937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.713217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.713955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:31.714263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:31.714714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.715378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.715649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:31.715735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.715922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.716692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:31.717003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:31.718368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:31.723711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 37/74] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:31.725510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.731444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 38/75] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:31.731444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.826289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3de3c-33a6-4394-ab76-0f5be7729143 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.826744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f1a87-0209-4657-98a8-2bf554e4d15c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.849877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.859563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba8a18-7bf5-4922-84ba-5056707b2c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.860403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba8a18-7bf5-4922-84ba-5056707b2c03 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.860691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba8a18-7bf5-4922-84ba-5056707b2c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.860790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:31.860968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba8a18-7bf5-4922-84ba-5056707b2c03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.861626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.861913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.862192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.868437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:31.872846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba8a18-7bf5-4922-84ba-5056707b2c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-eaba8a18-7bf5-4922-84ba-5056707b2c03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:31.881458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:31.882374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:31.883648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 39/76] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:31.884484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:31.884551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb787d2f-e472-49f2-9051-0569dd81e6ce None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:31.884551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb787d2f-e472-49f2-9051-0569dd81e6ce None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:31.884660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb787d2f-e472-49f2-9051-0569dd81e6ce None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:31.885361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:31.886015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 40/77] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:31.887089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:31.892097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.892210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb787d2f-e472-49f2-9051-0569dd81e6ce None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:31.987979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ba4bb-1381-4db6-95ee-8297834ff7fa None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:31.992507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb787d2f-e472-49f2-9051-0569dd81e6ce None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.022209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.023080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.023367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.023645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.024875 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.026036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c93f5a-fcb0-4c7d-a4a1-05f32cc814dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.026311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.026904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c93f5a-fcb0-4c7d-a4a1-05f32cc814dd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.027190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c93f5a-fcb0-4c7d-a4a1-05f32cc814dd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.027468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c93f5a-fcb0-4c7d-a4a1-05f32cc814dd None admin] PATH_INFO: `/v3/users/bd49b97115e3460492249a2e84a91b53` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.027935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 39/79] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:32.028721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.033041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c93f5a-fcb0-4c7d-a4a1-05f32cc814dd None admin] RBAC: Authorizing `identity:get_user(user_id=bd49b97115e3460492249a2e84a91b53)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.034564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c93f5a-fcb0-4c7d-a4a1-05f32cc814dd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.036245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 41/80] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:31 2022] GET /identity/v3/users/bd49b97115e3460492249a2e84a91b53 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:32.037969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34984d9a-cfe7-43c7-a4be-0a2f252e37eb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.038318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34984d9a-cfe7-43c7-a4be-0a2f252e37eb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.038598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34984d9a-cfe7-43c7-a4be-0a2f252e37eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.054661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-34984d9a-cfe7-43c7-a4be-0a2f252e37eb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:32.055841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-34984d9a-cfe7-43c7-a4be-0a2f252e37eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:32.081168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.118532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.124501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 42/81] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:32.125462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34984d9a-cfe7-43c7-a4be-0a2f252e37eb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.139348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590f117-f062-4199-a97f-2961c13502b4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.172635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.173309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.173590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.173859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.174594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:32.174898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.176266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.184754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 40/82] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:32.185528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.234929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34984d9a-cfe7-43c7-a4be-0a2f252e37eb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.270039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.270608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.270887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.271168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.285981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941e11f-8d80-40e9-8979-bb46f7abc340 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.298747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.320204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.320874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.321157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.321431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.335192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.339865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.341375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.388126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.389643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.438662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 43/83] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:32.439490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.502083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 41/84] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:32.502934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.518713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.545114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.549206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e57b9-ca2e-4298-99f1-157d74fffeb4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.587602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.588253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.588566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.588846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.609076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139739a8-8b21-4a19-aa88-5dd16abd00b1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.642974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.643766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.644058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.644338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.645550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.647574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.649226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 42/85] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:32.649985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.655808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.657290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.729769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.749339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91932e-4bfe-4d31-a2b6-9473e87f3fc1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.750131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.759317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 44/86] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:32.759317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.785993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.786868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.787172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.787547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.788815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.790256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.791877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 43/87] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:32.792665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.863111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b1367-091c-476c-8fc0-2378e73b9972 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.898006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.898876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.899163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.899450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.900669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.900767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341b898-0823-4ee6-ac42-7dfc95c6364e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:32.902133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.903844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 45/88] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:32.904612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.934197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:32.935056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:32.935336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:32.935616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] PATH_INFO: `/v3/projects/63f33c355ac74f32b40c55532d875391` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:32.936809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] RBAC: Authorizing `identity:get_project(project_id=63f33c355ac74f32b40c55532d875391)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:32.938236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:32.939917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 44/89] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/projects/63f33c355ac74f32b40c55532d875391 => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:32.940684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:32.958006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:32.974301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:33.008767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65466b-41b9-4e45-b818-7a44c9aa6e32 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.039788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03cfbf-2eae-4899-971e-8028d0286502 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.044013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.044715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.044998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.045274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.045984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.047429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.073629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.073847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 46/90] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:44:32 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:33.074578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.074950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.075234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.076619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.077296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:33.078080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.080050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 45/91] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:32 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:33.080775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:33.192604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:33.205394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809972e-5c90-4280-9c81-905e5acfbc62 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.213611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4afa02-bb61-4a99-a3a0-fa6cf5d83b49 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.233036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:33.249378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.250056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.250347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.250632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.253887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.255129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.255129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.255328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.256185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=841797717d6e4222a5d0793b3aeb3235) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:33.256539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.258009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.264678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 46/92] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:33 2022] GET /identity/v3/role_assignments?user.id=841797717d6e4222a5d0793b3aeb3235&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=98ebd7033f624d02bef09757068ee356 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:33.264678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:33.353121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.354875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.403502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfcfcd-4753-4d18-b236-b4779fda6de8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.453014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.453768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.454117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.454337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.455185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=63f33c355ac74f32b40c55532d875391, user.id=841797717d6e4222a5d0793b3aeb3235) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:33.455499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.458027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.466314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 47/93] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:33 2022] GET /identity/v3/role_assignments?user.id=841797717d6e4222a5d0793b3aeb3235&scope.project.id=63f33c355ac74f32b40c55532d875391&role.id=98ebd7033f624d02bef09757068ee356 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:33.467355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:33.496077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:33.500576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 47/94] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:33.506727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:33.542698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:33.637194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ea301-ee3a-4d05-8193-0d3880b74dc9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.662017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4340d-b9be-464d-8bed-84aab6987221 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.681857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.682808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.683238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.683758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.684772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=bd49b97115e3460492249a2e84a91b53) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:33.685262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.687034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.694551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 48/95] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:33 2022] GET /identity/v3/role_assignments?user.id=bd49b97115e3460492249a2e84a91b53&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:33.709709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cccc0-7232-4b65-877b-09f742354330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.710813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cccc0-7232-4b65-877b-09f742354330 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.711166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cccc0-7232-4b65-877b-09f742354330 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.711518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cccc0-7232-4b65-877b-09f742354330 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.713063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cccc0-7232-4b65-877b-09f742354330 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.714707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cccc0-7232-4b65-877b-09f742354330 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.716726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:33 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:33.722766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:33.834924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:33.870399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2e9b-c25e-46e5-b682-2706451df37f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:33.910394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84543831-258f-48d2-af7a-601dde861a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:33.911069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84543831-258f-48d2-af7a-601dde861a5a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:33.911360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84543831-258f-48d2-af7a-601dde861a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:33.911640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84543831-258f-48d2-af7a-601dde861a5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:33.913114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84543831-258f-48d2-af7a-601dde861a5a None admin] RBAC: Adding query filter params (role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=841797717d6e4222a5d0793b3aeb3235) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:33.913371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84543831-258f-48d2-af7a-601dde861a5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:33.914896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84543831-258f-48d2-af7a-601dde861a5a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:33.921460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:33 2022] GET /identity/v3/role_assignments?user.id=841797717d6e4222a5d0793b3aeb3235&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:36.170106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad703-e7af-4c33-acf7-18655d27c547 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.171095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad703-e7af-4c33-acf7-18655d27c547 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.171573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad703-e7af-4c33-acf7-18655d27c547 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.173186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:36.190326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312206d-6cee-47b9-ac33-e47a6dbd7404 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.191274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312206d-6cee-47b9-ac33-e47a6dbd7404 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.191752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312206d-6cee-47b9-ac33-e47a6dbd7404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.212933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b312206d-6cee-47b9-ac33-e47a6dbd7404 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:36.214469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b312206d-6cee-47b9-ac33-e47a6dbd7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:36.307365 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 50/99] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:36.344722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad703-e7af-4c33-acf7-18655d27c547 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:36.461386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:36.478118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad703-e7af-4c33-acf7-18655d27c547 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:36.523103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261660-494c-41e8-ab3a-615410f46fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:36.523910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261660-494c-41e8-ab3a-615410f46fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.524199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261660-494c-41e8-ab3a-615410f46fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.524510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261660-494c-41e8-ab3a-615410f46fa6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.525726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87261660-494c-41e8-ab3a-615410f46fa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:36.527190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87261660-494c-41e8-ab3a-615410f46fa6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:36.529182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 50/100] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:36.534768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312206d-6cee-47b9-ac33-e47a6dbd7404 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:36.629746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f644c7f-219c-4864-843a-8b9f39ec1dc7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.630033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f644c7f-219c-4864-843a-8b9f39ec1dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.630302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f644c7f-219c-4864-843a-8b9f39ec1dc7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.632456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 51/101] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:36.633494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:36.650284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144fa8-0f4d-4870-be76-903d12723c8e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.650574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144fa8-0f4d-4870-be76-903d12723c8e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.650851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144fa8-0f4d-4870-be76-903d12723c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.670398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312206d-6cee-47b9-ac33-e47a6dbd7404 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:36.694184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-af144fa8-0f4d-4870-be76-903d12723c8e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:36.695474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-af144fa8-0f4d-4870-be76-903d12723c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:36.722357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc853dd-0957-4c7d-8a7a-822fd69a4f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:36.722960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc853dd-0957-4c7d-8a7a-822fd69a4f98 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.723245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc853dd-0957-4c7d-8a7a-822fd69a4f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.723522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc853dd-0957-4c7d-8a7a-822fd69a4f98 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.724570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc853dd-0957-4c7d-8a7a-822fd69a4f98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:36.726073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc853dd-0957-4c7d-8a7a-822fd69a4f98 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:36.768771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 51/102] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:44:36 2022] POST /identity/v3/projects => generated 302 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:36.770260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf45fb-1ac5-44e2-944e-fb54b59aec9b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.770554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf45fb-1ac5-44e2-944e-fb54b59aec9b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.770841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf45fb-1ac5-44e2-944e-fb54b59aec9b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.777037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 52/103] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:36.786081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3adc-2caa-40ba-8ece-94f5508612eb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:36.786425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3adc-2caa-40ba-8ece-94f5508612eb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:36.786706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3adc-2caa-40ba-8ece-94f5508612eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:36.813718 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 52/104] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:36.814758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-608c3adc-2caa-40ba-8ece-94f5508612eb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:36.815979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-608c3adc-2caa-40ba-8ece-94f5508612eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:36.837722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af144fa8-0f4d-4870-be76-903d12723c8e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:36.981076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 53/105] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:37.033584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3adc-2caa-40ba-8ece-94f5508612eb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:37.048978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:37.070014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af144fa8-0f4d-4870-be76-903d12723c8e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:37.147525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e812ac-0baf-491f-90be-eaf8c2f90835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:37.148364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e812ac-0baf-491f-90be-eaf8c2f90835 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.148660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e812ac-0baf-491f-90be-eaf8c2f90835 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.148944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e812ac-0baf-491f-90be-eaf8c2f90835 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.150130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:37.173199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e812ac-0baf-491f-90be-eaf8c2f90835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-45e812ac-0baf-491f-90be-eaf8c2f90835 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:37.182597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:37.183478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:37.184228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:37.186679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 53/106] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:37.188165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad820-3adf-42e9-89f7-6e6eac3354b4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.188705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad820-3adf-42e9-89f7-6e6eac3354b4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.189192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad820-3adf-42e9-89f7-6e6eac3354b4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.190681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 54/107] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:37.196596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fad820-3adf-42e9-89f7-6e6eac3354b4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:37.225653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3adc-2caa-40ba-8ece-94f5508612eb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:37.331531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb06ae-5550-4e14-8eb2-0d12fa858b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:37.334004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb06ae-5550-4e14-8eb2-0d12fa858b8a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.334004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb06ae-5550-4e14-8eb2-0d12fa858b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.334004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb06ae-5550-4e14-8eb2-0d12fa858b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.350817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fad820-3adf-42e9-89f7-6e6eac3354b4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:37.376720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb06ae-5550-4e14-8eb2-0d12fa858b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5ecb06ae-5550-4e14-8eb2-0d12fa858b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:37.385671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:37.386555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:37.387303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:37.389879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 54/108] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:37.391461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888df2-005d-4da3-996c-89958d471068 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.391962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888df2-005d-4da3-996c-89958d471068 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.392480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b888df2-005d-4da3-996c-89958d471068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.412381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:37.412381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.412381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.412381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.412859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:37.412859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:37.414660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:37.429251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7b888df2-005d-4da3-996c-89958d471068 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:37.430163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 55/109] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:37.430953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7b888df2-005d-4da3-996c-89958d471068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:37.431057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:37.462274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:37.566220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1ab9e-4bc7-45bb-b60e-ac22650df60e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:37.590070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 55/110] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:37.591334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888df2-005d-4da3-996c-89958d471068 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:37.619633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e4d25-3467-4342-ac6f-b6ec8aa43c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:37.624381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e4d25-3467-4342-ac6f-b6ec8aa43c6d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.624381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e4d25-3467-4342-ac6f-b6ec8aa43c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.624381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e4d25-3467-4342-ac6f-b6ec8aa43c6d None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.624381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e4d25-3467-4342-ac6f-b6ec8aa43c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:37.624381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e4d25-3467-4342-ac6f-b6ec8aa43c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:37.624381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e4d25-3467-4342-ac6f-b6ec8aa43c6d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:37.646621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 56/111] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:37.649824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562d964-1572-4d9d-ae55-0ee0a283c36e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.650298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562d964-1572-4d9d-ae55-0ee0a283c36e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.650754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562d964-1572-4d9d-ae55-0ee0a283c36e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.655710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 57/112] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:37.660340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562d964-1572-4d9d-ae55-0ee0a283c36e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:37.671086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:37.744648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b888df2-005d-4da3-996c-89958d471068 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:37.777793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:37.792049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:37.793452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.794012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.794566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] PATH_INFO: `/v3/users/a19bd676fcaf4d7dace5c9f371705d30` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.812334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] RBAC: Authorizing `identity:get_user(user_id=a19bd676fcaf4d7dace5c9f371705d30)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:37.813844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:37.815862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 56/113] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:37 2022] GET /identity/v3/users/a19bd676fcaf4d7dace5c9f371705d30 => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:37.816345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:37.826583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562d964-1572-4d9d-ae55-0ee0a283c36e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:37.864969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf114d-42aa-46c0-8a7d-0250471f252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:37.865843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf114d-42aa-46c0-8a7d-0250471f252a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.866327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf114d-42aa-46c0-8a7d-0250471f252a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.866608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf114d-42aa-46c0-8a7d-0250471f252a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.877023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf114d-42aa-46c0-8a7d-0250471f252a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-3ddf114d-42aa-46c0-8a7d-0250471f252a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:37.886336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:37.887242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:37.888023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 58/114] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:44:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:37.889380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64543c3f-70c3-4fd4-a346-f1f1a172b7e1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.889380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64543c3f-70c3-4fd4-a346-f1f1a172b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.889380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64543c3f-70c3-4fd4-a346-f1f1a172b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.906212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-64543c3f-70c3-4fd4-a346-f1f1a172b7e1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:37.907413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-64543c3f-70c3-4fd4-a346-f1f1a172b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:37.925373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:37.929972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0553a-97f3-4d05-8223-1fbcca50e331 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:37.971269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db01ff9-482f-40e4-a498-abdc005b69d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:37.972174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01ff9-482f-40e4-a498-abdc005b69d4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.972484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01ff9-482f-40e4-a498-abdc005b69d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.972761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01ff9-482f-40e4-a498-abdc005b69d4 None admin] PATH_INFO: `/v3/users/bd49b97115e3460492249a2e84a91b53` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.975053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db01ff9-482f-40e4-a498-abdc005b69d4 None admin] RBAC: Authorizing `identity:get_user(user_id=bd49b97115e3460492249a2e84a91b53)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:37.976661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db01ff9-482f-40e4-a498-abdc005b69d4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:37.982679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 57/115] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:37 2022] GET /identity/v3/users/bd49b97115e3460492249a2e84a91b53 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:37.982679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ecc93-3210-4ed4-a0d5-100892556a2a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:37.982679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ecc93-3210-4ed4-a0d5-100892556a2a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:37.982679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ecc93-3210-4ed4-a0d5-100892556a2a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:37.982679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 58/116] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:37.982679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ecc93-3210-4ed4-a0d5-100892556a2a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.012991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 59/117] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:38.014139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64543c3f-70c3-4fd4-a346-f1f1a172b7e1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.078648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.104956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ecc93-3210-4ed4-a0d5-100892556a2a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.121787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64543c3f-70c3-4fd4-a346-f1f1a172b7e1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.144063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae27bd5-2374-45a5-b1d8-c3f0788385e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.147860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae27bd5-2374-45a5-b1d8-c3f0788385e1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.147860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae27bd5-2374-45a5-b1d8-c3f0788385e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.147860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae27bd5-2374-45a5-b1d8-c3f0788385e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.166092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.166763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.167044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.167316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.168059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:38.168401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:38.169765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:38.178662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 60/118] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:38.179383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.190584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.227245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae27bd5-2374-45a5-b1d8-c3f0788385e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:38.228749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae27bd5-2374-45a5-b1d8-c3f0788385e1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:38.294789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e892dc-5a21-4fe3-b60b-a5f9dc1faff2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.322667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.335229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 59/119] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:38.336440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9ff7c-be3c-48cc-bdac-fc3b5277de57 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.336734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9ff7c-be3c-48cc-bdac-fc3b5277de57 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.337011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9ff7c-be3c-48cc-bdac-fc3b5277de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.342482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.343629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.343919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.344193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.345429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:38.346881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:38.348670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 61/120] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:38.350531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.360887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-edc9ff7c-be3c-48cc-bdac-fc3b5277de57 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:38.362079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-edc9ff7c-be3c-48cc-bdac-fc3b5277de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:38.424120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.430076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 60/121] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:38.430890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9ff7c-be3c-48cc-bdac-fc3b5277de57 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.478146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6dfc3-21d2-430b-a546-87bd2e3fb17f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.515220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.516013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.516301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.516680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.529166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:38.536921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:38.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:38.538528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:38.539095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 62/122] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:38.541422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.547804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9ff7c-be3c-48cc-bdac-fc3b5277de57 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.586687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.587906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.588037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.588208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.589465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:38.590926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:38.596795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 61/123] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:38.596795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.605684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.658921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb85aa-56ab-4718-8932-a39c063d476d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.660817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.694752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.695858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.696426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.696712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.698010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:38.698114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde39638-eb60-4b57-ac83-22e6d534172c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.699637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:38.702424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 63/124] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:38.703012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.733284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.734188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.734472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.734756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/bd49b97115e3460492249a2e84a91b53/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.736728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f41ab15737f42cd803bf7f27bc66b36, user_id=bd49b97115e3460492249a2e84a91b53, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:38.738221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:38.760388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 62/125] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:38 2022] PUT /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/bd49b97115e3460492249a2e84a91b53/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:44:38.760388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.840595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb91ce2-bc15-4dce-b76a-96a377738f62 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.865615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.884038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.884983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.885342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.885715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.893810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bef22-4565-48af-9e63-5e8c009976e2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:38.899500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:38.908637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:38.909572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:38.910345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 64/126] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:38.910964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:38.929962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:38.945393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:38.946691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:38.947310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:38.947693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:38.948838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:38.949680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:38.951491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:38.964102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 63/127] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:38.965013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.043710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf3605-a81c-499c-9b36-9f8bdca2ce93 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.078964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.079900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.080112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.080416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.081615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.083077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.085821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 65/128] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:39.086403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.113346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b46417-3099-40ad-8a3b-8432c0b65e76 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.156015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:39.165049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.167100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.167718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.168237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.169514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] RBAC: Adding query filter params (role.id=f4351ca9183346af85c1667531e247fb, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=a19bd676fcaf4d7dace5c9f371705d30) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:39.170018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.172229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.181459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 64/129] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:38 2022] GET /identity/v3/role_assignments?user.id=a19bd676fcaf4d7dace5c9f371705d30&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=f4351ca9183346af85c1667531e247fb => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:39.182660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.252405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffcc1b-509a-4ae5-9517-0af5063b58fb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.282765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:39.347964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.350819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.351323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.352729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.353851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:39.355783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.357425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.375472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 66/130] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:39.377444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.390564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c110-ece7-4ab2-a44b-10e4d2e6dd46 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.447845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.448996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.449466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.449907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.452851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.454969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.457830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:39 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:39.459124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.530043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1601f90-f9fe-4b5a-bd7f-f6ea349ec81f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.542074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:39.569448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.570673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.571094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.571589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/841797717d6e4222a5d0793b3aeb3235/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.573984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f41ab15737f42cd803bf7f27bc66b36, user_id=841797717d6e4222a5d0793b3aeb3235, role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.575616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.588973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4164-7c8c-41dd-8c44-7281e8ab1247 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.594210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 67/132] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:39 2022] PUT /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/841797717d6e4222a5d0793b3aeb3235/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:44:39.595333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.626218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:39.656900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b9fe6-81cf-4cda-bc06-c065e5363038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.657813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b9fe6-81cf-4cda-bc06-c065e5363038 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.658164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b9fe6-81cf-4cda-bc06-c065e5363038 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.658451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b9fe6-81cf-4cda-bc06-c065e5363038 None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.665592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b9fe6-81cf-4cda-bc06-c065e5363038 None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.666922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b9fe6-81cf-4cda-bc06-c065e5363038 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.668943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 66/133] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:39 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:39.670149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30c5a3-9332-4ae2-a994-aa12935afc1f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.670443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30c5a3-9332-4ae2-a994-aa12935afc1f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.670731 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30c5a3-9332-4ae2-a994-aa12935afc1f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.672034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 67/134] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:39.674879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30c5a3-9332-4ae2-a994-aa12935afc1f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.746373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77065e-3283-4626-9cda-b28eb56d7056 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.816691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6a28-5aaa-490f-a300-c6eb036cc859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.817592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6a28-5aaa-490f-a300-c6eb036cc859 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.817881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6a28-5aaa-490f-a300-c6eb036cc859 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.818322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6a28-5aaa-490f-a300-c6eb036cc859 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.820196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d6a28-5aaa-490f-a300-c6eb036cc859 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.821716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d6a28-5aaa-490f-a300-c6eb036cc859 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.827169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 68/135] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:39 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:39.833799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fbc8-840e-42b7-a004-4235bd6f03a6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.834571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fbc8-840e-42b7-a004-4235bd6f03a6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.835199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fbc8-840e-42b7-a004-4235bd6f03a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.875280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5407fbc8-840e-42b7-a004-4235bd6f03a6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:39.876514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5407fbc8-840e-42b7-a004-4235bd6f03a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:39.882481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30c5a3-9332-4ae2-a994-aa12935afc1f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:39.915537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:39.941360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:39.942622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:39.943131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:39.944234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] PATH_INFO: `/v3/projects/63f33c355ac74f32b40c55532d875391` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:39.947408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] RBAC: Authorizing `identity:get_project(project_id=63f33c355ac74f32b40c55532d875391)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:39.949204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:39.951326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 68/136] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:39 2022] GET /identity/v3/projects/63f33c355ac74f32b40c55532d875391 => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:39.952258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:39.976656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 69/137] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:39.977906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407fbc8-840e-42b7-a004-4235bd6f03a6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:40.063310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:40.149281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d0e92-4f7d-460a-86e7-bc6d9cfdb96d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:40.169043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407fbc8-840e-42b7-a004-4235bd6f03a6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:40.216948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:40.217856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:40.218140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:40.218416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] PATH_INFO: `/v3/projects/63f33c355ac74f32b40c55532d875391/users/841797717d6e4222a5d0793b3aeb3235/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:40.220436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] RBAC: Authorizing `identity:create_grant(project_id=63f33c355ac74f32b40c55532d875391, user_id=841797717d6e4222a5d0793b3aeb3235, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:40.221895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:40.221993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:40.222882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:40.223170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:40.223448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/841797717d6e4222a5d0793b3aeb3235/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:40.229849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f41ab15737f42cd803bf7f27bc66b36, user_id=841797717d6e4222a5d0793b3aeb3235, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:40.234022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:40.241083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 70/138] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:39 2022] PUT /identity/v3/projects/63f33c355ac74f32b40c55532d875391/users/841797717d6e4222a5d0793b3aeb3235/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:44:40.248746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:40.270415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 69/139] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:39 2022] PUT /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/841797717d6e4222a5d0793b3aeb3235/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:44:40.299905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:40.384606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae775c-e3bf-4ef5-b755-dd5d4f044f77 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:40.424325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6f765-12a8-4f90-a480-698f86454823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:40.425092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6f765-12a8-4f90-a480-698f86454823 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:40.425376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6f765-12a8-4f90-a480-698f86454823 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:40.425652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6f765-12a8-4f90-a480-698f86454823 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:40.427083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6f765-12a8-4f90-a480-698f86454823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:40.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6f765-12a8-4f90-a480-698f86454823 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:40.430372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 71/140] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:40.435928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:40.571840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f4e4b-1342-4bc0-803c-33ce6bfe42a1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:40.590274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:40.610531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fef0c-93a2-495a-908d-c83fb2ef10eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:40.611245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fef0c-93a2-495a-908d-c83fb2ef10eb None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:40.611529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fef0c-93a2-495a-908d-c83fb2ef10eb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:40.611806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fef0c-93a2-495a-908d-c83fb2ef10eb None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:40.612571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fef0c-93a2-495a-908d-c83fb2ef10eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:40.614031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fef0c-93a2-495a-908d-c83fb2ef10eb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:40.642596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 70/141] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:44:40 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:41.482843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b851-b84d-4c4a-81d0-2cf39c3753ff None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:41.483321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b851-b84d-4c4a-81d0-2cf39c3753ff None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:41.483321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b851-b84d-4c4a-81d0-2cf39c3753ff None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:41.485013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 72/142] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:41.491146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208081f-9383-46ce-84d9-783788846b25 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:41.492041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208081f-9383-46ce-84d9-783788846b25 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:41.492562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208081f-9383-46ce-84d9-783788846b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:41.510881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d208081f-9383-46ce-84d9-783788846b25 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:41.512396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d208081f-9383-46ce-84d9-783788846b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:41.591689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 71/143] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:41.633823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501b851-b84d-4c4a-81d0-2cf39c3753ff None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:41.744528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:41.753225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501b851-b84d-4c4a-81d0-2cf39c3753ff None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:41.791470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:41.792677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:41.793163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:41.793636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:41.805498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:41.813691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:41.814622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:41.815372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:41.817735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 73/144] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:41.824199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208081f-9383-46ce-84d9-783788846b25 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:41.961204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208081f-9383-46ce-84d9-783788846b25 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:41.962717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:42.001756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31e590-ed73-41e1-bc68-363751568a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:42.002770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e590-ed73-41e1-bc68-363751568a2a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.003256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e590-ed73-41e1-bc68-363751568a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.003723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e590-ed73-41e1-bc68-363751568a2a None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.004723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e590-ed73-41e1-bc68-363751568a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:42.005243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e590-ed73-41e1-bc68-363751568a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:42.007258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e590-ed73-41e1-bc68-363751568a2a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:42.020169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 72/145] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:42.026497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:42.149855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460538ae-989c-43e7-8e38-be19a922fd1a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:42.159080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a959-1a34-48c2-bf45-479c0e53d231 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.159655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a959-1a34-48c2-bf45-479c0e53d231 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.160134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a959-1a34-48c2-bf45-479c0e53d231 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.161732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 73/146] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:42.167590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3be812-5b38-490a-b12c-4790ef8ce6d9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.168140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3be812-5b38-490a-b12c-4790ef8ce6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.168636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3be812-5b38-490a-b12c-4790ef8ce6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.197338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3f3be812-5b38-490a-b12c-4790ef8ce6d9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:42.198549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3f3be812-5b38-490a-b12c-4790ef8ce6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:42.222698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:42.224250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.225299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.225721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] PATH_INFO: `/v3/users/bd49b97115e3460492249a2e84a91b53` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.227201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] RBAC: Authorizing `identity:get_user(user_id=bd49b97115e3460492249a2e84a91b53)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:42.228883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:42.230787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 74/147] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:42 2022] GET /identity/v3/users/bd49b97115e3460492249a2e84a91b53 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:42.253052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:42.292010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 74/148] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:42.302155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:42.319169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3be812-5b38-490a-b12c-4790ef8ce6d9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:42.428469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:42.436241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3be812-5b38-490a-b12c-4790ef8ce6d9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:42.436782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0e89b-ca96-41aa-970f-ac5bc744e09f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:42.482765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:42.483632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.483919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.484201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.486387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:42.487046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.487327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.487604 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.510487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:42.526246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:42.527131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:42.527887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 75/149] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:42.535018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:42.635102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:42.636889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:42.675923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:42.687055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4faff7-5f0f-4827-98c6-7b748ac762b9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:42.726267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:42.727182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.727640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.728061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.728989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:42.729549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:42.731186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:42.741526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 76/150] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:42.747536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:42.768495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:42.780158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 75/151] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 15:44:42.798439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:42.872205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0614d-fe75-40d1-92e9-eb5ce6b415dc None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:42.911579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:42.913331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:42.913885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:42.914314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] PATH_INFO: `/v3/users/a19bd676fcaf4d7dace5c9f371705d30` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:42.915705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] RBAC: Authorizing `identity:get_user(user_id=a19bd676fcaf4d7dace5c9f371705d30)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:42.917461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:42.923376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 77/152] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:42 2022] GET /identity/v3/users/a19bd676fcaf4d7dace5c9f371705d30 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:42.925922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:43.023492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874c387-8c02-4107-a283-8dd4f1bc3fa5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:43.030049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:43.056695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37709c5a-f98d-4e84-9629-db549cac632c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:43.093418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:43.106898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:43.108073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.108524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.108962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.110327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:43.111928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:43.118370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 78/153] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:42 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:43.124724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:43.133128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e75d5-2110-4c40-a9fd-b505446bc8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:43.134014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e75d5-2110-4c40-a9fd-b505446bc8fa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.134291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e75d5-2110-4c40-a9fd-b505446bc8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.139998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e75d5-2110-4c40-a9fd-b505446bc8fa None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.141113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e75d5-2110-4c40-a9fd-b505446bc8fa None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:43.142942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e75d5-2110-4c40-a9fd-b505446bc8fa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:43.144330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 76/154] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:42 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:43.145630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37959f92-06f1-4e3a-a979-786c46bdc133 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.145853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37959f92-06f1-4e3a-a979-786c46bdc133 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.146129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37959f92-06f1-4e3a-a979-786c46bdc133 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.151546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 77/155] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:43.153737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111623e5-c341-423f-a50f-e2648b14838d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.154022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111623e5-c341-423f-a50f-e2648b14838d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.154301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111623e5-c341-423f-a50f-e2648b14838d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.217384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-111623e5-c341-423f-a50f-e2648b14838d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:43.218586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-111623e5-c341-423f-a50f-e2648b14838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:43.281015 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81291a-1877-407a-9a70-d4dcc6167e17 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:43.319563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:43.321294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.321755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.322178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/a19bd676fcaf4d7dace5c9f371705d30/roles/f4351ca9183346af85c1667531e247fb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.325085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f41ab15737f42cd803bf7f27bc66b36, user_id=a19bd676fcaf4d7dace5c9f371705d30, role_id=f4351ca9183346af85c1667531e247fb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:43.327660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:43.346799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 78/156] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:43.348228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111623e5-c341-423f-a50f-e2648b14838d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:43.352589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 79/157] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:43 2022] PUT /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/users/a19bd676fcaf4d7dace5c9f371705d30/roles/f4351ca9183346af85c1667531e247fb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:44:43.383805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:43.409621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:43.456016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111623e5-c341-423f-a50f-e2648b14838d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:43.492401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:43.515075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:43.515742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.516022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.516293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.520403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=bd49b97115e3460492249a2e84a91b53) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:43.520403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:43.520403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:43.530401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 79/158] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:43 2022] GET /identity/v3/role_assignments?user.id=bd49b97115e3460492249a2e84a91b53&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:43.572195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bcf42-231e-43a4-94f2-8c4757f1a1d9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:43.631943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:43.633201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.633763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.634235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.656656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:43.666657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:43.667593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:43.668419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:43.670479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 80/159] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:44:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:43.676632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:43.807628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8b161-e9c1-4aeb-8069-23e13e0d6878 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:43.844098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:43.844730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:43.845006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:43.845278 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:43.846037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:43.846339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:43.847735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:43.857972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 80/160] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:44:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:43.863567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:43.939938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:43.976623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13100af-77b9-4270-8b1d-9ca7760c73a3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:44.018675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:44.019556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:44.019831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:44.020117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:44.021966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:44.023420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:44.025104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 81/161] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:43 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:44.030683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:44.085586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:44.138723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d99e20-9bb2-4319-9f5e-67af1eb1ce24 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:44.173740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:44.174393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:44.174670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:44.174939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:44.247107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:44.248594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:44.297440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:44.352239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 81/162] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:44.357517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:44.477349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f028830-7ddf-4442-9c4b-94e1d7acd4f8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:44.523152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c57a7-6fe3-4c63-80c7-00a7deec8fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:44.524021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c57a7-6fe3-4c63-80c7-00a7deec8fad None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:44.524294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c57a7-6fe3-4c63-80c7-00a7deec8fad None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:44.524590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c57a7-6fe3-4c63-80c7-00a7deec8fad None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:44.528194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c57a7-6fe3-4c63-80c7-00a7deec8fad None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:44.529721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c57a7-6fe3-4c63-80c7-00a7deec8fad None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:44.531683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 82/163] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:44 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:44.537964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:44.641376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ec4b1-7b57-4b90-a549-4c895a2c7f4a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:44.676456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40a98d-8745-44e3-8fd1-3f32a09bd3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:44.677116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40a98d-8745-44e3-8fd1-3f32a09bd3ab None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:44.677512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40a98d-8745-44e3-8fd1-3f32a09bd3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:44.678041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40a98d-8745-44e3-8fd1-3f32a09bd3ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:44.678783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40a98d-8745-44e3-8fd1-3f32a09bd3ab None admin] RBAC: Adding query filter params (role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=841797717d6e4222a5d0793b3aeb3235) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:44.679092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40a98d-8745-44e3-8fd1-3f32a09bd3ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:44.680562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40a98d-8745-44e3-8fd1-3f32a09bd3ab None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:44.687033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 82/164] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:44 2022] GET /identity/v3/role_assignments?user.id=841797717d6e4222a5d0793b3aeb3235&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:45.130247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfcffb-83f1-49ee-b35d-a2d757abf9a5 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.132104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfcffb-83f1-49ee-b35d-a2d757abf9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.132631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfcffb-83f1-49ee-b35d-a2d757abf9a5 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.134532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 83/165] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:45.145979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338d7f1-fa05-42e1-90f9-b16c942cde20 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.147975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338d7f1-fa05-42e1-90f9-b16c942cde20 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.148509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338d7f1-fa05-42e1-90f9-b16c942cde20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.172609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7338d7f1-fa05-42e1-90f9-b16c942cde20 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:45.174138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7338d7f1-fa05-42e1-90f9-b16c942cde20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:45.186677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:45.255931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 83/166] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:45.295692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfcffb-83f1-49ee-b35d-a2d757abf9a5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:45.319921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:45.415503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba901e7-9d97-4be8-a4dd-1b8091a6065d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.415831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba901e7-9d97-4be8-a4dd-1b8091a6065d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.416071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba901e7-9d97-4be8-a4dd-1b8091a6065d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.417721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 84/167] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:45.424258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e332d-6b1e-436b-8ca0-989e55a4c806 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.424732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e332d-6b1e-436b-8ca0-989e55a4c806 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.425019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e332d-6b1e-436b-8ca0-989e55a4c806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.461922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-332e332d-6b1e-436b-8ca0-989e55a4c806 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:45.465124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-332e332d-6b1e-436b-8ca0-989e55a4c806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:45.470411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfcffb-83f1-49ee-b35d-a2d757abf9a5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:45.548064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2780b1-9319-4803-8846-2ebd694533ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:45.548894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2780b1-9319-4803-8846-2ebd694533ff None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.549177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2780b1-9319-4803-8846-2ebd694533ff None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.549450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2780b1-9319-4803-8846-2ebd694533ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.566494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2780b1-9319-4803-8846-2ebd694533ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5d2780b1-9319-4803-8846-2ebd694533ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:45.576185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:45.577292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:45.578358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 84/168] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:45.579151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053edc54-20bb-459d-bc2a-9860cde3ff47 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.579278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053edc54-20bb-459d-bc2a-9860cde3ff47 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.579562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053edc54-20bb-459d-bc2a-9860cde3ff47 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.586031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 85/169] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:45.588158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053edc54-20bb-459d-bc2a-9860cde3ff47 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:45.620186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 85/170] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:45.622180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdcab4-1a18-47e7-a09c-a4a6bd1750ba None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.622605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdcab4-1a18-47e7-a09c-a4a6bd1750ba None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.623069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdcab4-1a18-47e7-a09c-a4a6bd1750ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.673720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-91bdcab4-1a18-47e7-a09c-a4a6bd1750ba None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:45.674844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-91bdcab4-1a18-47e7-a09c-a4a6bd1750ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:45.720773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:45.768224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053edc54-20bb-459d-bc2a-9860cde3ff47 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:45.813063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 86/171] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:45.814218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdcab4-1a18-47e7-a09c-a4a6bd1750ba None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:45.822497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:45.889059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:45.889748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.890029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.890304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:45.891056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:45.891359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:45.892762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:45.914747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 86/172] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:45.915592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:45.950813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdcab4-1a18-47e7-a09c-a4a6bd1750ba None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:45.987302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:45.988124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:45.988455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:45.988735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.001354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:46.010173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:46.011106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:46.012372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 87/173] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:46.017563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.084689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:46.111710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf49f76-e5b9-486b-a462-7b3ecbcfceb6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.123070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4ff0c-3a76-4b49-b667-d05143a086c1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.144404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:46.152612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.155508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.156183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.156738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.158409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.159439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.159796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.160146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.162481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:46.164099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:46.166219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 88/174] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:46.166940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.173791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:46.183312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:46.184261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:46.185078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:46.188085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 87/175] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:46.189193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.283581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3016d-5d3d-4ab1-b98f-327cbe745ce5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.301003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6a179-7f1f-49c4-806f-58e0186065f0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.314314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:46.328382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.328382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.328382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.328382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.328973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:46.328973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:46.328973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:46.336031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 89/176] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:46.341362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.371780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.372523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.372896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.373223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.444611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe03311-7d7d-4b69-89a1-f2b5bffc78b7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.480256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.480979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.481260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.481537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.482292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:46.482598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:46.483984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:46.492705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:46.493712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 90/177] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:46.494579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.512793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:46.514422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:46.579886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:46.635939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25ffbd-e6a3-4e77-8450-6b18199ba10f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.646529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 88/178] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:46.647569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.680321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3dcd1-7a17-46bd-9ec6-a953a48a87cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.681332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dcd1-7a17-46bd-9ec6-a953a48a87cc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.681688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dcd1-7a17-46bd-9ec6-a953a48a87cc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.682038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dcd1-7a17-46bd-9ec6-a953a48a87cc None admin] PATH_INFO: `/v3/users/841797717d6e4222a5d0793b3aeb3235` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.683302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3dcd1-7a17-46bd-9ec6-a953a48a87cc None admin] RBAC: Authorizing `identity:get_user(user_id=841797717d6e4222a5d0793b3aeb3235)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:46.684882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3dcd1-7a17-46bd-9ec6-a953a48a87cc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:46.686710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 91/179] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/users/841797717d6e4222a5d0793b3aeb3235 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:46.692063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780a75-058f-4210-8a69-87fa06617690 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.692366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780a75-058f-4210-8a69-87fa06617690 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.692651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780a75-058f-4210-8a69-87fa06617690 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.695174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 92/180] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:46.695765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780a75-058f-4210-8a69-87fa06617690 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.751101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:46.761784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2c476-6d0e-447a-a707-52a9db219403 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.796002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f078f-199f-470c-ac02-f569d3ceb493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.796909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f078f-199f-470c-ac02-f569d3ceb493 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.797191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f078f-199f-470c-ac02-f569d3ceb493 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.797463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f078f-199f-470c-ac02-f569d3ceb493 None admin] PATH_INFO: `/v3/users/24f8dbc8cf4341bf9b524f44de5cf606` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.808060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780a75-058f-4210-8a69-87fa06617690 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.810611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f078f-199f-470c-ac02-f569d3ceb493 None admin] RBAC: Authorizing `identity:get_user(user_id=24f8dbc8cf4341bf9b524f44de5cf606)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:46.812074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f078f-199f-470c-ac02-f569d3ceb493 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:46.813798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 89/181] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/users/24f8dbc8cf4341bf9b524f44de5cf606 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:46.815108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba66a2-0678-4d4f-ab46-226b4b4587b8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.815393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba66a2-0678-4d4f-ab46-226b4b4587b8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.815667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba66a2-0678-4d4f-ab46-226b4b4587b8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.821303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 90/182] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:46.821303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba66a2-0678-4d4f-ab46-226b4b4587b8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.839400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:46.852861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30edb73-ee1c-4721-ad4b-33e0ca157330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.853721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30edb73-ee1c-4721-ad4b-33e0ca157330 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.854001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30edb73-ee1c-4721-ad4b-33e0ca157330 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.854276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30edb73-ee1c-4721-ad4b-33e0ca157330 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.855705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30edb73-ee1c-4721-ad4b-33e0ca157330 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:46.857160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30edb73-ee1c-4721-ad4b-33e0ca157330 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:46.858831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 93/183] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:46.859955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b33a2-e679-4ea8-82c3-d09ff75adfb3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.860234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b33a2-e679-4ea8-82c3-d09ff75adfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.860530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b33a2-e679-4ea8-82c3-d09ff75adfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.876182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-eb4b33a2-e679-4ea8-82c3-d09ff75adfb3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:46.877368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eb4b33a2-e679-4ea8-82c3-d09ff75adfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:46.923670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba66a2-0678-4d4f-ab46-226b4b4587b8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:46.945321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 94/184] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:46.946145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b33a2-e679-4ea8-82c3-d09ff75adfb3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:46.970736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630ab62-0198-4631-b733-3d8eb16789c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:46.971389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ab62-0198-4631-b733-3d8eb16789c5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:46.971668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ab62-0198-4631-b733-3d8eb16789c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:46.971943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ab62-0198-4631-b733-3d8eb16789c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:46.993032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.043229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630ab62-0198-4631-b733-3d8eb16789c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:47.044719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630ab62-0198-4631-b733-3d8eb16789c5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:47.047576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b33a2-e679-4ea8-82c3-d09ff75adfb3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.068879 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.083209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.084051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.084393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.084676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.149889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 91/185] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:47.151179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283e076-843b-4831-979e-5f74c6671a30 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.151468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283e076-843b-4831-979e-5f74c6671a30 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.151739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283e076-843b-4831-979e-5f74c6671a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.156560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:47.158039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:47.167718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0283e076-843b-4831-979e-5f74c6671a30 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:47.168921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0283e076-843b-4831-979e-5f74c6671a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:47.213268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.236642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 92/186] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:47.238161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc0f7-890a-42fe-8d97-be9009be826e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.238448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc0f7-890a-42fe-8d97-be9009be826e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.238727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc0f7-890a-42fe-8d97-be9009be826e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.240894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 93/187] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:47.241305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc0f7-890a-42fe-8d97-be9009be826e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.269038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 95/188] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:47.269877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.290110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.352598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc0f7-890a-42fe-8d97-be9009be826e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.375876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c92202-6d00-4dbf-a4e1-25703193085b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.385978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.386631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.386911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.387190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.388012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=841797717d6e4222a5d0793b3aeb3235) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:47.388323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:47.389804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:47.395628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 94/189] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/role_assignments?user.id=841797717d6e4222a5d0793b3aeb3235&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=98ebd7033f624d02bef09757068ee356 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:47.396905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.421761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7832f-b4c8-4a7a-9feb-90b9f083c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.422641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7832f-b4c8-4a7a-9feb-90b9f083c403 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.422921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7832f-b4c8-4a7a-9feb-90b9f083c403 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.423188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7832f-b4c8-4a7a-9feb-90b9f083c403 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.424481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7832f-b4c8-4a7a-9feb-90b9f083c403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:47.426140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7832f-b4c8-4a7a-9feb-90b9f083c403 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:47.428008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 96/190] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:47.431022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9125115-8e54-4c23-9d33-1ccc3deb91ce None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.431022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9125115-8e54-4c23-9d33-1ccc3deb91ce None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.431022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9125115-8e54-4c23-9d33-1ccc3deb91ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.448692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f9125115-8e54-4c23-9d33-1ccc3deb91ce None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:47.449911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f9125115-8e54-4c23-9d33-1ccc3deb91ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:47.455314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.520305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7226309-3b64-4759-8324-8e94e7347dd8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.528913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 97/191] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:47.537229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9125115-8e54-4c23-9d33-1ccc3deb91ce None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.549864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.571161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.572037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.572314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.572628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] PATH_INFO: `/v3/projects/63f33c355ac74f32b40c55532d875391` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.573859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] RBAC: Authorizing `identity:get_project(project_id=63f33c355ac74f32b40c55532d875391)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:47.575354 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:47.577327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 95/192] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/projects/63f33c355ac74f32b40c55532d875391 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:47.582887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.641295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9125115-8e54-4c23-9d33-1ccc3deb91ce None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.683170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.684025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.684393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.684679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.694786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:47.696618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588314fe-0508-4283-b12d-0569b4e915da None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:47.703513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:47.704471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:47.705239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 98/193] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:47.705878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.710504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.735145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.736030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.736309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.736620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.737795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:47.739236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:47.741625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 96/194] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:47.742378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.792696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:47.817449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cbeb-6f55-4f35-b8e2-24516924eb99 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.842096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d929ad-05d3-4584-915f-7503644caf8f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:47.851706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.852395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.852683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.852960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.853673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:47.855112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:47.876531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 99/195] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:44:47 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:47.877374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.877974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:47.878798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:47.879084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:47.879357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:47.891719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:47.899723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:47.900924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 97/196] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:47.905243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:47.950022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:48.005351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa53d-86eb-4ffd-a092-4fbe5a86c0e2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.022499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10879b49-107c-491a-92b4-84b542a1eeae None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.044732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.045401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.045683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.045964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.046807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=63f33c355ac74f32b40c55532d875391, user.id=841797717d6e4222a5d0793b3aeb3235) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:48.047116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.048702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.050810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:48.063671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.064390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.064681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.064958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.065729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:48.066035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.069104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.069275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 100/197] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/role_assignments?user.id=841797717d6e4222a5d0793b3aeb3235&scope.project.id=63f33c355ac74f32b40c55532d875391&role.id=98ebd7033f624d02bef09757068ee356 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:48.070253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:48.081865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 98/198] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:44:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:48.085421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:48.222813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac59910-9a38-4ca8-ba7d-65ce11f172f9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.241155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:48.244134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021503ce-5ec2-4482-89b2-e5162fca30c0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.270873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.272487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.273065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.273648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.274820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=24f8dbc8cf4341bf9b524f44de5cf606) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:48.275361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.277204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.285425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 101/199] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:48 2022] GET /identity/v3/role_assignments?user.id=24f8dbc8cf4341bf9b524f44de5cf606&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:48.286292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.286466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:48.288253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.288861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.289311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.290266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:48.290739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.292071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.301811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 99/200] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:48.310242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:48.372842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:48.413347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e45d8-be45-4b8e-90e6-23897cfb531e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.445048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e0fb-e77e-45e9-be6d-288dbdc1dad8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.461323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.462324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.462610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.462887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] PATH_INFO: `/v3/users/24f8dbc8cf4341bf9b524f44de5cf606` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.464178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] RBAC: Authorizing `identity:get_user(user_id=24f8dbc8cf4341bf9b524f44de5cf606)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.465676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.467786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 102/201] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:48 2022] GET /identity/v3/users/24f8dbc8cf4341bf9b524f44de5cf606 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:48.476854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:48.503476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.504483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.504775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.505050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] PATH_INFO: `/v3/users/a19bd676fcaf4d7dace5c9f371705d30` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.510506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=a19bd676fcaf4d7dace5c9f371705d30)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.512002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.515474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 100/202] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:48 2022] GET /identity/v3/users/a19bd676fcaf4d7dace5c9f371705d30 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:48.521842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:48.583836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:48.617214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd9bf9-f49b-48fa-b9e2-8fc98e742a1d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.648117 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:48.659225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.659875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.660215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.660446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.736507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5320876-eed5-47dc-9157-5e6ec6dfd3e7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:48.795664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.797157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.801106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:48.801764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:48.802040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:48.802372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:48.909903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:48.913884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:48.956378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 103/203] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 15:44:48.963109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:48.981750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:48.997835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:49.075456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dfa86-ede6-4fe2-9f3a-a596d54b6a3f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:49.114384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:49.115956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:49.116492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:49.116867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:49.118111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:49.119575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:49.121626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 104/204] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:48 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:49.128229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:49.239288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 101/205] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:49.248884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e92dc-d89c-4895-95dc-da46c93ad4ea None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:49.249411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:49.252730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:49.290348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:49.291322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:49.291920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:49.292562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:49.293616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] RBAC: Adding query filter params (role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=24f8dbc8cf4341bf9b524f44de5cf606) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:49.294281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:49.296220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:49.303503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 105/206] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:49 2022] GET /identity/v3/role_assignments?user.id=24f8dbc8cf4341bf9b524f44de5cf606&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:49.456823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92041db-2e65-4d06-90f6-209ae8474782 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:49.508811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:49.515061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0eac5-cd09-486a-9e50-4b5cad5a0b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:49.515942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0eac5-cd09-486a-9e50-4b5cad5a0b5c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:49.516229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0eac5-cd09-486a-9e50-4b5cad5a0b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:49.516691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0eac5-cd09-486a-9e50-4b5cad5a0b5c None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:49.521824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0eac5-cd09-486a-9e50-4b5cad5a0b5c None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:49.523309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0eac5-cd09-486a-9e50-4b5cad5a0b5c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:49.525535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 102/207] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:49 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:49.531905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:49.690146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6091f50-cb6b-4a81-b4d5-7b3f10223575 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:49.766101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d58032-f110-4d93-8a1d-813dd7d43137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:49.766701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d58032-f110-4d93-8a1d-813dd7d43137 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:49.766981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d58032-f110-4d93-8a1d-813dd7d43137 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:49.767254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d58032-f110-4d93-8a1d-813dd7d43137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:49.768097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d58032-f110-4d93-8a1d-813dd7d43137 None admin] RBAC: Adding query filter params (role.id=f4351ca9183346af85c1667531e247fb, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36, user.id=a19bd676fcaf4d7dace5c9f371705d30) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:49.768430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d58032-f110-4d93-8a1d-813dd7d43137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:49.769894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d58032-f110-4d93-8a1d-813dd7d43137 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:49.788104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 106/208] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:49 2022] GET /identity/v3/role_assignments?user.id=a19bd676fcaf4d7dace5c9f371705d30&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=f4351ca9183346af85c1667531e247fb => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:50.686806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d36d9-83c0-48ce-b8aa-fd863213b02b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:50.687345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d36d9-83c0-48ce-b8aa-fd863213b02b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:50.687613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d36d9-83c0-48ce-b8aa-fd863213b02b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:50.689014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 103/209] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:50.694803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83eaef-2ca8-4185-b0a9-2548c5c1f535 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:50.695694 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83eaef-2ca8-4185-b0a9-2548c5c1f535 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:50.696175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83eaef-2ca8-4185-b0a9-2548c5c1f535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:50.714532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5e83eaef-2ca8-4185-b0a9-2548c5c1f535 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:50.716148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e83eaef-2ca8-4185-b0a9-2548c5c1f535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:50.741957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:50.792028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 107/210] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:50.815631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d36d9-83c0-48ce-b8aa-fd863213b02b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:50.855139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38a1c8-99a6-48c0-abd2-68afee916841 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:50.855464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38a1c8-99a6-48c0-abd2-68afee916841 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:50.855700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38a1c8-99a6-48c0-abd2-68afee916841 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:50.858383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 108/211] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:50.867659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa212d-2f0a-4f83-9532-100a93ee8222 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:50.868072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa212d-2f0a-4f83-9532-100a93ee8222 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:50.868562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa212d-2f0a-4f83-9532-100a93ee8222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:50.887190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-00fa212d-2f0a-4f83-9532-100a93ee8222 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:50.888579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-00fa212d-2f0a-4f83-9532-100a93ee8222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:50.926508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d36d9-83c0-48ce-b8aa-fd863213b02b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:50.963063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:50.963269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 109/212] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:50.964392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:50.964850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:50.965284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:50.976652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:50.984590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:50.985515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:50.986333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:50.988198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa212d-2f0a-4f83-9532-100a93ee8222 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:50.989003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 104/213] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:50.998140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.063678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.084332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.100926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa212d-2f0a-4f83-9532-100a93ee8222 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.112793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733e81e-28e2-47fe-ae47-dd666fa1828e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.141236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da50a9c-1cf3-43b9-9ce6-cf7362474a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.142051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50a9c-1cf3-43b9-9ce6-cf7362474a6e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.142330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50a9c-1cf3-43b9-9ce6-cf7362474a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.142603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50a9c-1cf3-43b9-9ce6-cf7362474a6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.154430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3703bec-638d-4f78-94ed-9da7c0220951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.155183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3703bec-638d-4f78-94ed-9da7c0220951 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.155454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3703bec-638d-4f78-94ed-9da7c0220951 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.155667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3703bec-638d-4f78-94ed-9da7c0220951 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.155919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da50a9c-1cf3-43b9-9ce6-cf7362474a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:51.156467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3703bec-638d-4f78-94ed-9da7c0220951 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:51.156873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3703bec-638d-4f78-94ed-9da7c0220951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.158269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3703bec-638d-4f78-94ed-9da7c0220951 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5da50a9c-1cf3-43b9-9ce6-cf7362474a6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:51.164028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:51.164942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:51.165735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 110/214] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:51.166970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe29fb0-fe22-49d5-a0f1-df831d379309 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.166970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe29fb0-fe22-49d5-a0f1-df831d379309 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.167135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe29fb0-fe22-49d5-a0f1-df831d379309 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.168571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 111/215] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:51.172674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 105/216] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:51.173213 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe29fb0-fe22-49d5-a0f1-df831d379309 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.178763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4093d-95a2-4375-a611-d043c2b61c9d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.179050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4093d-95a2-4375-a611-d043c2b61c9d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.179330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4093d-95a2-4375-a611-d043c2b61c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.197623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9ae4093d-95a2-4375-a611-d043c2b61c9d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:51.198828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9ae4093d-95a2-4375-a611-d043c2b61c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:51.272544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 106/217] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:51.272544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4093d-95a2-4375-a611-d043c2b61c9d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.291385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe29fb0-fe22-49d5-a0f1-df831d379309 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.326752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.327423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.327709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.327985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.328789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:51.329098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.330480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.338885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.340541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 112/218] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:51.346785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.363167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.405275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4093d-95a2-4375-a611-d043c2b61c9d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.439765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.440745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.441025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.441296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] PATH_INFO: `/v3/users/24f8dbc8cf4341bf9b524f44de5cf606` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.442490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] RBAC: Authorizing `identity:get_user(user_id=24f8dbc8cf4341bf9b524f44de5cf606)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.444566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.446207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 107/219] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:51 2022] GET /identity/v3/users/24f8dbc8cf4341bf9b524f44de5cf606 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:51.446959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.467993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1bbf-3995-4bde-b3aa-8a85309cfb47 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.504707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.505572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.505853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.506125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.507673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.509144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.510928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 113/220] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:51.511607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.563060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd4784-ccbb-4afc-b5f7-cda536267d9d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.581948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.596021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.601898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.602972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.603410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.603692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] PATH_INFO: `/v3/users/70c728b2ecc441c1a62dc2b649ceaf39` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.618707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] RBAC: Authorizing `identity:get_user(user_id=70c728b2ecc441c1a62dc2b649ceaf39)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.620163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.622078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 108/221] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:51 2022] GET /identity/v3/users/70c728b2ecc441c1a62dc2b649ceaf39 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:51.622642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.623681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd528-ae21-4d32-b24d-416818fa9440 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.663201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.664485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.664968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.665430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.666873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.668590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.675513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 114/222] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:51 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:51.676691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.724987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101ceb-9c87-442c-a273-1d18a59949a7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.759825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.760523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.760802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.761076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.761772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.763194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.775828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dc386-01a9-4399-b534-8985c4374533 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.785950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 109/223] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:44:51 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:51.786816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:51.812798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.813533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.813755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.814023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.832528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.840039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:51.900108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.902189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.924831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc6ea4-47d3-46be-841b-a25fd2d3d9ba None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:51.965923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7217799-dd22-498b-9f67-c1940ba84adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:51.967005 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217799-dd22-498b-9f67-c1940ba84adc None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.967657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217799-dd22-498b-9f67-c1940ba84adc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.967939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217799-dd22-498b-9f67-c1940ba84adc None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777/users/24f8dbc8cf4341bf9b524f44de5cf606/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.970223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217799-dd22-498b-9f67-c1940ba84adc None admin] RBAC: Authorizing `identity:create_grant(project_id=c109cd55dcce43668065e0c553a39777, user_id=24f8dbc8cf4341bf9b524f44de5cf606, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:51.971701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217799-dd22-498b-9f67-c1940ba84adc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:51.986924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 110/224] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:51 2022] PUT /identity/v3/projects/c109cd55dcce43668065e0c553a39777/users/24f8dbc8cf4341bf9b524f44de5cf606/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:44:51.988003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ebb96-ee18-4ec0-bc35-7a6df18e1406 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.988295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ebb96-ee18-4ec0-bc35-7a6df18e1406 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.988595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ebb96-ee18-4ec0-bc35-7a6df18e1406 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:51.990097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 111/225] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:51.995994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21e929-f2d0-448f-97c5-cb9e9fc15dd7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:51.996293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21e929-f2d0-448f-97c5-cb9e9fc15dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:51.996608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21e929-f2d0-448f-97c5-cb9e9fc15dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:52.017285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ec21e929-f2d0-448f-97c5-cb9e9fc15dd7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:52.020108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ec21e929-f2d0-448f-97c5-cb9e9fc15dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:52.030549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 115/226] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:52.038203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:52.096081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:52.127711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 112/227] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:52.170814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21e929-f2d0-448f-97c5-cb9e9fc15dd7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:52.176315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a948-fdfa-41ae-bc53-bc7aceeb68e9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:52.206578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:52.225891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:52.226781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:52.227065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:52.227332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:52.229511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:52.231034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:52.232837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 116/228] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:52 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:52.239797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:52.314151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21e929-f2d0-448f-97c5-cb9e9fc15dd7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:52.359650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:52.361290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:52.362065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:52.362555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:52.363966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6c2cc-68e8-4e78-8f91-4e866a3e1448 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:52.375939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:52.379542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:52.387926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:52.388883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:52.389667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:52.396516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 113/229] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:44:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:44:52.399396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:52.410473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:52.411512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:52.411512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:52.412374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:52.413229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=70c728b2ecc441c1a62dc2b649ceaf39) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:52.413592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:52.415298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:52.422385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 117/230] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:52 2022] GET /identity/v3/role_assignments?user.id=70c728b2ecc441c1a62dc2b649ceaf39&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=98ebd7033f624d02bef09757068ee356 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:52.497356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:52.521921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba246cf0-4a45-481e-8d67-3a291367e85c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:52.561864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:52.563152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:52.563152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:52.563152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:52.563848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:52.564153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:52.565588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:52.580401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 114/231] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:44:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:52.582019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:52.690633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9538c-d0bc-4d94-afa5-14e476c6f5b1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:52.727802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:52.728708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:52.728989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:52.729262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] PATH_INFO: `/v3/users/24f8dbc8cf4341bf9b524f44de5cf606` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:52.730455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] RBAC: Authorizing `identity:get_user(user_id=24f8dbc8cf4341bf9b524f44de5cf606)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:52.731901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:52.733711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 118/232] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:52 2022] GET /identity/v3/users/24f8dbc8cf4341bf9b524f44de5cf606 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:52.739211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:52.851336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064b8e-bf33-4835-a4cb-f303a42a8bfc None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:52.887723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d9541-0bdd-4f79-a329-5d9b3960e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:52.888728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9541-0bdd-4f79-a329-5d9b3960e4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:52.889007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9541-0bdd-4f79-a329-5d9b3960e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:52.889277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9541-0bdd-4f79-a329-5d9b3960e4c7 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:52.890474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d9541-0bdd-4f79-a329-5d9b3960e4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:52.891914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d9541-0bdd-4f79-a329-5d9b3960e4c7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:52.893814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 115/233] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:52 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:52.899978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:52.965549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:53.033765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b28094-b147-4f96-b9d6-ac91a2c59924 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:53.086436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9017d8-38b3-47c6-b81e-98cfb788d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:53.087395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9017d8-38b3-47c6-b81e-98cfb788d90c None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:53.087672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9017d8-38b3-47c6-b81e-98cfb788d90c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:53.087941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9017d8-38b3-47c6-b81e-98cfb788d90c None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777/users/24f8dbc8cf4341bf9b524f44de5cf606/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:53.090280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9017d8-38b3-47c6-b81e-98cfb788d90c None admin] RBAC: Authorizing `identity:create_grant(project_id=c109cd55dcce43668065e0c553a39777, user_id=24f8dbc8cf4341bf9b524f44de5cf606, role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:53.091706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9017d8-38b3-47c6-b81e-98cfb788d90c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:53.121531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 119/234] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:52 2022] PUT /identity/v3/projects/c109cd55dcce43668065e0c553a39777/users/24f8dbc8cf4341bf9b524f44de5cf606/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:44:54.664095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507dcf-66b5-4d93-b985-45b6832fcf10 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:54.664648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507dcf-66b5-4d93-b985-45b6832fcf10 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:54.664648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507dcf-66b5-4d93-b985-45b6832fcf10 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:54.666305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 116/235] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:54.671894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d74a9-8249-4a10-b5ac-474f172099f9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:54.672290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d74a9-8249-4a10-b5ac-474f172099f9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:54.672487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d74a9-8249-4a10-b5ac-474f172099f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:54.690353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-561d74a9-8249-4a10-b5ac-474f172099f9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:54.691552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-561d74a9-8249-4a10-b5ac-474f172099f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:54.761471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 120/236] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:54.784864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507dcf-66b5-4d93-b985-45b6832fcf10 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:54.801649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:54.824398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4cd8e-1364-47f6-9bae-39cc71731ef9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:54.824679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4cd8e-1364-47f6-9bae-39cc71731ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:54.824954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4cd8e-1364-47f6-9bae-39cc71731ef9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:54.826675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 121/237] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:54.832170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af545c-e176-4132-8412-5da263247ac8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:54.832506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af545c-e176-4132-8412-5da263247ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:54.832782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af545c-e176-4132-8412-5da263247ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:54.853616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-25af545c-e176-4132-8412-5da263247ac8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:54.855243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-25af545c-e176-4132-8412-5da263247ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:54.894484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:54.936546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 122/238] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:44:54.982045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507dcf-66b5-4d93-b985-45b6832fcf10 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:54.995218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af545c-e176-4132-8412-5da263247ac8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.020755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacbc9a-57b7-4382-8289-8825fa19505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.022002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbc9a-57b7-4382-8289-8825fa19505b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.022478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbc9a-57b7-4382-8289-8825fa19505b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.022944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbc9a-57b7-4382-8289-8825fa19505b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.034885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacbc9a-57b7-4382-8289-8825fa19505b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-cbacbc9a-57b7-4382-8289-8825fa19505b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:55.043524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:55.044780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:44:55.045581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:55.048187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 117/239] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:44:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:55.049844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9d8a4-9b07-4359-b58f-40588d1014ac None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.050387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9d8a4-9b07-4359-b58f-40588d1014ac None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.050924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9d8a4-9b07-4359-b58f-40588d1014ac None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.053124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 118/240] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:55.058908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9d8a4-9b07-4359-b58f-40588d1014ac None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.129867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af545c-e176-4132-8412-5da263247ac8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:55.136263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:55.173722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddcf7b-cc4f-4236-ac45-536719413402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.173928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9d8a4-9b07-4359-b58f-40588d1014ac None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:55.174544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddcf7b-cc4f-4236-ac45-536719413402 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.174823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddcf7b-cc4f-4236-ac45-536719413402 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.175093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddcf7b-cc4f-4236-ac45-536719413402 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.176439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddcf7b-cc4f-4236-ac45-536719413402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:55.178739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddcf7b-cc4f-4236-ac45-536719413402 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:55.180386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 123/241] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:55.185505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a2435-9ed6-43cd-bf90-633ec903c813 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.187716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a2435-9ed6-43cd-bf90-633ec903c813 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.188148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a2435-9ed6-43cd-bf90-633ec903c813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.205141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ff7a2435-9ed6-43cd-bf90-633ec903c813 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:55.206312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ff7a2435-9ed6-43cd-bf90-633ec903c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:55.220010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.221231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.222108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.222852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.223838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:55.224767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:55.226155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:55.243938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 119/242] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:55.243938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.247159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:55.312428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 124/243] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:55.313949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a2435-9ed6-43cd-bf90-633ec903c813 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.380019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf7fae-0d6b-482d-b36a-6862e58fa50e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:55.416912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.417591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.417872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.418145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.418848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:55.420308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:55.426960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:55.441208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a2435-9ed6-43cd-bf90-633ec903c813 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:55.448737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 120/244] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:44:55 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:44:55.448737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.498878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1736f-100a-444f-9eb4-13b6b6f169d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.499767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1736f-100a-444f-9eb4-13b6b6f169d7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.500048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1736f-100a-444f-9eb4-13b6b6f169d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.500318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1736f-100a-444f-9eb4-13b6b6f169d7 None admin] PATH_INFO: `/v3/users/24f8dbc8cf4341bf9b524f44de5cf606` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.501566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1736f-100a-444f-9eb4-13b6b6f169d7 None admin] RBAC: Authorizing `identity:get_user(user_id=24f8dbc8cf4341bf9b524f44de5cf606)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:55.503026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1736f-100a-444f-9eb4-13b6b6f169d7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:55.505150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 125/245] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:55 2022] GET /identity/v3/users/24f8dbc8cf4341bf9b524f44de5cf606 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:55.507217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1479c0-6182-4ac7-b8e3-64cd4ace3690 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.507500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1479c0-6182-4ac7-b8e3-64cd4ace3690 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.507773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1479c0-6182-4ac7-b8e3-64cd4ace3690 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.513526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 126/246] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:55.514925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2157f-c275-4ba9-ba2b-3f197d6bae1a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.515210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2157f-c275-4ba9-ba2b-3f197d6bae1a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.515484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2157f-c275-4ba9-ba2b-3f197d6bae1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.533875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-84d2157f-c275-4ba9-ba2b-3f197d6bae1a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:55.535080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-84d2157f-c275-4ba9-ba2b-3f197d6bae1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:55.563939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:55.597543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fceb8-098c-4b99-90a6-c4902f067aa2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:55.617154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 127/247] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:55.625261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2157f-c275-4ba9-ba2b-3f197d6bae1a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.647004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.647821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.648100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.652094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.665441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:55.673274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:55.674264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:55.675357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 121/248] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:55.678346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.747544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:55.770005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2157f-c275-4ba9-ba2b-3f197d6bae1a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:55.809071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e25cb-8ce8-4818-808d-eb8b561f9a46 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:55.809868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.810690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.811037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.811419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.852468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:55.853285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:55.853570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:55.853848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:55.860592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:55.869541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:55.877785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:55.878675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:55.879443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 122/249] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:55.880076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:55.905803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:55.907596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:55.987915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f2bb9-1a79-458a-96ea-6e30d39d08a6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.001987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:56.020836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 128/250] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:56.021996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.034668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.035647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.036249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.037182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.038002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:56.038373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:56.039943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:56.050896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 123/251] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:56.052010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.098034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:56.160986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608c9ed-bb4f-483f-a24e-153d0ad17789 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.165029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b90ea-cc36-453f-a936-a6b8bb2e1c03 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.204976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a298-4902-400f-8c53-c1ddab55c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.206003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a298-4902-400f-8c53-c1ddab55c427 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.206475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a298-4902-400f-8c53-c1ddab55c427 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.206939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a298-4902-400f-8c53-c1ddab55c427 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.207043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a4917-b7ff-4663-a478-af94f4fa3ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.207891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a4917-b7ff-4663-a478-af94f4fa3ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.208177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a4917-b7ff-4663-a478-af94f4fa3ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.208421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84a298-4902-400f-8c53-c1ddab55c427 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:56.209154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a4917-b7ff-4663-a478-af94f4fa3ce5 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.209622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84a298-4902-400f-8c53-c1ddab55c427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:56.210465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a4917-b7ff-4663-a478-af94f4fa3ce5 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:56.211381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84a298-4902-400f-8c53-c1ddab55c427 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:56.211912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a4917-b7ff-4663-a478-af94f4fa3ce5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:56.217992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 124/252] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:56.217992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b00420-e9b2-43be-bf49-20c252a81a2a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.217992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b00420-e9b2-43be-bf49-20c252a81a2a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.217992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b00420-e9b2-43be-bf49-20c252a81a2a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.217992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 125/253] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:56.218551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b00420-e9b2-43be-bf49-20c252a81a2a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.229336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 129/254] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:56.231062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34850211-d833-4926-9c6c-241f09f112ca None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.231572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34850211-d833-4926-9c6c-241f09f112ca None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.232052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34850211-d833-4926-9c6c-241f09f112ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.251499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-34850211-d833-4926-9c6c-241f09f112ca None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:56.253142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-34850211-d833-4926-9c6c-241f09f112ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:56.274542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:56.339115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 130/255] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:56.340466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34850211-d833-4926-9c6c-241f09f112ca None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.366553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b00420-e9b2-43be-bf49-20c252a81a2a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.408686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.409567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.409847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.410118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] PATH_INFO: `/v3/users/e0998e47fd0a492eb19475cdba8f9d30` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.415553 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:56.429025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=e0998e47fd0a492eb19475cdba8f9d30)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:56.430486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:56.432438 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 126/256] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/users/e0998e47fd0a492eb19475cdba8f9d30 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:56.432995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.488886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34850211-d833-4926-9c6c-241f09f112ca None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.531683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.532144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.532468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.532788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.533688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=24f8dbc8cf4341bf9b524f44de5cf606) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:56.533925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:56.535492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:56.547159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 131/257] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/role_assignments?user.id=24f8dbc8cf4341bf9b524f44de5cf606&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:56.547159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.566434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:56.573658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823b4e7-d45c-4cc9-8db8-5ada27729ba3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.645195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.647372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.647664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.647943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] PATH_INFO: `/v3/users/70c728b2ecc441c1a62dc2b649ceaf39` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.649220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] RBAC: Authorizing `identity:get_user(user_id=70c728b2ecc441c1a62dc2b649ceaf39)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:56.650687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:56.652844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 127/258] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/users/70c728b2ecc441c1a62dc2b649ceaf39 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:56.653702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.743932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867721-1172-468b-bc69-39f21f443ae4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.784830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:56.812462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.813276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.813555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.813827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.826446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:56.833706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e8d1b-5335-40ed-820a-94cd64978c1c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:56.836347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:56.837282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 132/259] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:56.838506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:56.903922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:56.904609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:56.904886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:56.905157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:56.955673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:56.989299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0418adb-7137-4ddf-b118-3161e1b7960f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:57.006725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.008196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.034027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:57.034949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:57.035172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:57.035446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:57.036691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.038139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.040153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 133/260] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:57.040958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:57.109578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:57.142050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 128/261] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:44:57.142866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:57.189186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f350c7-d2cb-4183-bfde-e17588164167 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:57.222745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:57.224806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:57.225799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:57.226363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:57.226363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:57.227248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:57.227557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.228976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.242501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 134/262] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:44:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:57.242501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:57.269602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8bd3c-6916-472d-8a6c-08ba0d36373e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:57.307261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:57.308611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:57.308611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:57.308961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777/users/70c728b2ecc441c1a62dc2b649ceaf39/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:57.311018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c109cd55dcce43668065e0c553a39777, user_id=70c728b2ecc441c1a62dc2b649ceaf39, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.312605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.335737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 129/263] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:44:57 2022] PUT /identity/v3/projects/c109cd55dcce43668065e0c553a39777/users/70c728b2ecc441c1a62dc2b649ceaf39/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:44:57.336873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:57.396958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d063ed-6b01-4e89-b5a1-5d94151958a3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:57.420935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:57.452468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:57.453870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:57.454310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:57.455279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:57.456876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.459243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.461611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 135/264] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:57 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:57.469013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:57.587171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a3e1d-37a9-4aa5-988c-f960857456f8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:57.596396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055d52-ef9f-4549-8f7d-9bed5a82b948 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:57.641184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:57.642037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:57.643472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:57.644037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:57.644519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:57.645811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] RBAC: Adding query filter params (role.id=f4351ca9183346af85c1667531e247fb, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=e0998e47fd0a492eb19475cdba8f9d30) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:57.646326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.648233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.655126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:57.656137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:57.656732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 136/265] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:57 2022] GET /identity/v3/role_assignments?user.id=e0998e47fd0a492eb19475cdba8f9d30&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=f4351ca9183346af85c1667531e247fb => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:57.656866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:57.660382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] PATH_INFO: `/v3/users/24f8dbc8cf4341bf9b524f44de5cf606` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:57.660382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] RBAC: Authorizing `identity:get_user(user_id=24f8dbc8cf4341bf9b524f44de5cf606)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.660382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.663410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 130/266] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:57 2022] GET /identity/v3/users/24f8dbc8cf4341bf9b524f44de5cf606 => generated 283 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:57.669148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:57.714296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:57.794486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11db7e-bc80-4007-9a89-554b3ca9b16b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:57.847320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4060316-227a-4585-a158-f72713e92d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:57.847901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4060316-227a-4585-a158-f72713e92d44 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:57.848185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4060316-227a-4585-a158-f72713e92d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:57.848534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4060316-227a-4585-a158-f72713e92d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:57.950646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4060316-227a-4585-a158-f72713e92d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:57.952146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4060316-227a-4585-a158-f72713e92d44 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:57.994874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:58.063909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 137/267] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 15:44:58.070716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:58.090045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5376-4be9-4a05-ace5-3427086ff4aa None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.090339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5376-4be9-4a05-ace5-3427086ff4aa None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.090598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5376-4be9-4a05-ace5-3427086ff4aa None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.092115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 138/268] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:44:58.097868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e594-2c93-4fd8-807b-cde0da5b297e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.098088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e594-2c93-4fd8-807b-cde0da5b297e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.098365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e594-2c93-4fd8-807b-cde0da5b297e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.118338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-59a8e594-2c93-4fd8-807b-cde0da5b297e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:58.119510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-59a8e594-2c93-4fd8-807b-cde0da5b297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:58.203599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c323f15-edc8-4dbf-9a2a-a960a76bc9e1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:58.222454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 139/269] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:58.244676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:58.245543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.245825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.246098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.247280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:58.248745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:58.250730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 131/270] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:44:58 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:58.256637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e594-2c93-4fd8-807b-cde0da5b297e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:58.268584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:58.383047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:58.383872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:58.426921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e594-2c93-4fd8-807b-cde0da5b297e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:58.436821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8729b-fb91-4351-9649-42cc51797ea3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:58.464435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed9aba-572c-4733-b317-df8cb7ec4ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:58.465361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed9aba-572c-4733-b317-df8cb7ec4ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.465786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed9aba-572c-4733-b317-df8cb7ec4ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.466205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed9aba-572c-4733-b317-df8cb7ec4ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.467178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed9aba-572c-4733-b317-df8cb7ec4ba7 None admin] RBAC: Adding query filter params (role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=24f8dbc8cf4341bf9b524f44de5cf606) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:58.467639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed9aba-572c-4733-b317-df8cb7ec4ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:58.469277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed9aba-572c-4733-b317-df8cb7ec4ba7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:58.474404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:58.475214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.475497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.475769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.477494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 140/271] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:44:58 2022] GET /identity/v3/role_assignments?user.id=24f8dbc8cf4341bf9b524f44de5cf606&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:58.488074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4735218-0ec8-40f0-8817-33e165481d43 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.488074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4735218-0ec8-40f0-8817-33e165481d43 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.488074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4735218-0ec8-40f0-8817-33e165481d43 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.488074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 141/272] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:44:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:44:58.495780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d9fd-4c5e-401d-a995-e33bf5daa88c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.496094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d9fd-4c5e-401d-a995-e33bf5daa88c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.496338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d9fd-4c5e-401d-a995-e33bf5daa88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.499103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:44:58.507764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:44:58.508762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:44:58.509560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:44:58.518093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 132/273] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:44:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:44:58.520928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:58.535623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0304d9fd-4c5e-401d-a995-e33bf5daa88c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:44:58.537779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0304d9fd-4c5e-401d-a995-e33bf5daa88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:44:58.685838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 142/274] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:44:58.691320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09676a8-2e71-4028-82b2-5afa77a44fa1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:58.711692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304d9fd-4c5e-401d-a995-e33bf5daa88c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:58.728390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:58.746328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:58.747010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.747291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.747568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.748319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:58.748667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:58.750055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:58.763251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 133/275] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:44:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:58.768385 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:58.796040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:58.844445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304d9fd-4c5e-401d-a995-e33bf5daa88c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:58.888989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161cf3b-d91a-4e50-a8cc-f84e177d34ee None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:58.889210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:58.890170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.890706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.890983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.892597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:58.897765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:58.901669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 143/276] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:44:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:44:58.906477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:58.933508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:58.934769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:58.935277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:58.935798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] PATH_INFO: `/v3/users/0d6bc8ccc5514f6dad6f030230692c0a` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:58.954220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=0d6bc8ccc5514f6dad6f030230692c0a)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:58.955705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:58.957928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 134/277] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:44:58 2022] GET /identity/v3/users/0d6bc8ccc5514f6dad6f030230692c0a => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:44:58.969390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:44:59.026357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:59.037139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696781b9-0a0b-431f-af16-d156dcaf54cd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:59.070331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:44:59.072636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe469d-b80a-4c53-a9d0-8bb29351cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:59.073441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe469d-b80a-4c53-a9d0-8bb29351cdc7 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:59.073728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe469d-b80a-4c53-a9d0-8bb29351cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:59.074003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe469d-b80a-4c53-a9d0-8bb29351cdc7 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:59.074720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe469d-b80a-4c53-a9d0-8bb29351cdc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:59.076267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe469d-b80a-4c53-a9d0-8bb29351cdc7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:59.098189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 144/278] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:44:58 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:44:59.104434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629803f2-e85f-4179-814a-5ece3d3af7c2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:44:59.155824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5830c9-b243-45c9-ad73-6b379a4489b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:44:59.156963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5830c9-b243-45c9-ad73-6b379a4489b8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:44:59.157474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5830c9-b243-45c9-ad73-6b379a4489b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:44:59.157926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5830c9-b243-45c9-ad73-6b379a4489b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:44:59.158976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5830c9-b243-45c9-ad73-6b379a4489b8 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, user.id=0d6bc8ccc5514f6dad6f030230692c0a, scope.system=all) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:44:59.159529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5830c9-b243-45c9-ad73-6b379a4489b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:44:59.161272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5830c9-b243-45c9-ad73-6b379a4489b8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:44:59.167968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 135/279] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Jun 7 15:44:58 2022] GET /identity/v3/role_assignments?user.id=0d6bc8ccc5514f6dad6f030230692c0a&scope.system=all&role.id=98ebd7033f624d02bef09757068ee356 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:00.197773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603cea4-ce21-4e9a-b81b-43cb323db77c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:00.198299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603cea4-ce21-4e9a-b81b-43cb323db77c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:00.198299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603cea4-ce21-4e9a-b81b-43cb323db77c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:00.199709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 145/280] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:00.205815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e729-beef-4e51-8f20-6818c3ae1924 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:00.206327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e729-beef-4e51-8f20-6818c3ae1924 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:00.206399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e729-beef-4e51-8f20-6818c3ae1924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:00.225864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0895e729-beef-4e51-8f20-6818c3ae1924 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:00.227211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0895e729-beef-4e51-8f20-6818c3ae1924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:00.303783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 136/281] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:00.346146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603cea4-ce21-4e9a-b81b-43cb323db77c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:00.502078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603cea4-ce21-4e9a-b81b-43cb323db77c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:00.555749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2fe22-b5a2-48a9-afc2-a4d6a3e1ab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:00.556631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2fe22-b5a2-48a9-afc2-a4d6a3e1ab5a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:00.556954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2fe22-b5a2-48a9-afc2-a4d6a3e1ab5a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:00.557244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2fe22-b5a2-48a9-afc2-a4d6a3e1ab5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:00.570443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2fe22-b5a2-48a9-afc2-a4d6a3e1ab5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-45d2fe22-b5a2-48a9-afc2-a4d6a3e1ab5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:00.578185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:00.579067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:00.579841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 146/282] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:00.585570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895e729-beef-4e51-8f20-6818c3ae1924 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:00.633799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6baa-b941-42c8-8982-02301fabd189 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:00.634121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6baa-b941-42c8-8982-02301fabd189 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:00.634362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6baa-b941-42c8-8982-02301fabd189 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:00.635886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 147/283] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:00.638445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:00.643525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b3c3b-acad-4ce6-9168-7cd33ec2b207 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:00.643832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b3c3b-acad-4ce6-9168-7cd33ec2b207 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:00.644096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b3c3b-acad-4ce6-9168-7cd33ec2b207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:00.675698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ec8b3c3b-acad-4ce6-9168-7cd33ec2b207 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:00.676936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ec8b3c3b-acad-4ce6-9168-7cd33ec2b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:00.719391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895e729-beef-4e51-8f20-6818c3ae1924 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:00.750316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:00.762874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:00.763833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:00.764319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:00.764814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:00.765743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:00.766231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:00.767885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:00.778964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 137/284] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:00.784283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:00.791229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 148/285] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:00.813546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b3c3b-acad-4ce6-9168-7cd33ec2b207 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:00.895594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:00.900975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa492e4-0173-4662-bfda-c82f474b74d6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:00.941801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:00.943585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:00.944100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:00.944579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] PATH_INFO: `/v3/users/70c728b2ecc441c1a62dc2b649ceaf39` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:00.946453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] RBAC: Authorizing `identity:get_user(user_id=70c728b2ecc441c1a62dc2b649ceaf39)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:00.948058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:00.950133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 138/286] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:00 2022] GET /identity/v3/users/70c728b2ecc441c1a62dc2b649ceaf39 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:00.955451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:01.065471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c73b1-9435-4686-8e75-a583efe8b10a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:01.077874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b3c3b-acad-4ce6-9168-7cd33ec2b207 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:01.120091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:01.120785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.121069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.121347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.122000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:01.122795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.123072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.123341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.147495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:01.166094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:01.167014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:01.167809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 149/287] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:01.176457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:01.179997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:01.216676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:01.219764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:01.257021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:01.331695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 139/288] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 15:45:01.341570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:01.396290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02607c4-b1f5-4fc6-b702-6c89bc4d74f2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:01.469922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:01.470619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:01.470903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.471184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.471452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.472209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:01.472538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:01.473954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:01.494911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 150/289] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:01.500629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:01.532282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e6e6-f170-4234-89a6-49e13e161723 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:01.570989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:01.571859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.572143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.572448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.573676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:01.575226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:01.577266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 140/290] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:01 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:01.582738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:01.731273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:01.737768 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c052ff0-aeed-4893-9ab8-75f381a72e5a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:01.786951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63334d-76ed-4d16-a0e6-d9735d985916 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:01.795399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:01.796310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.796624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.796897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] PATH_INFO: `/v3/users/e0998e47fd0a492eb19475cdba8f9d30` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.798196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] RBAC: Authorizing `identity:get_user(user_id=e0998e47fd0a492eb19475cdba8f9d30)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:01.799639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:01.802045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 151/291] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:01 2022] GET /identity/v3/users/e0998e47fd0a492eb19475cdba8f9d30 => generated 297 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:01.807433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:01.860077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e610b-c7b8-40ea-9b78-f68f59155b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:01.860908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e610b-c7b8-40ea-9b78-f68f59155b20 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.861186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e610b-c7b8-40ea-9b78-f68f59155b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.861454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e610b-c7b8-40ea-9b78-f68f59155b20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.862276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e610b-c7b8-40ea-9b78-f68f59155b20 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=70c728b2ecc441c1a62dc2b649ceaf39) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:01.862579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e610b-c7b8-40ea-9b78-f68f59155b20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:01.864006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e610b-c7b8-40ea-9b78-f68f59155b20 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:01.873794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 141/292] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:01 2022] GET /identity/v3/role_assignments?user.id=70c728b2ecc441c1a62dc2b649ceaf39&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=98ebd7033f624d02bef09757068ee356 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:01.873794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5485a7-3bc7-4641-99b5-1e565a3a09d7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.873794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5485a7-3bc7-4641-99b5-1e565a3a09d7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.873794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5485a7-3bc7-4641-99b5-1e565a3a09d7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.873794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 142/293] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:01.874679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:01.894112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354e3f8-7fae-46d7-b672-a8b56af47e92 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:01.894738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354e3f8-7fae-46d7-b672-a8b56af47e92 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:01.895233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354e3f8-7fae-46d7-b672-a8b56af47e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:01.917206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5354e3f8-7fae-46d7-b672-a8b56af47e92 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:01.919664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5354e3f8-7fae-46d7-b672-a8b56af47e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:02.009419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 143/294] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:02.013674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccee4b-47b4-41f9-9ca7-dbd188ed225e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.014184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccee4b-47b4-41f9-9ca7-dbd188ed225e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.014675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccee4b-47b4-41f9-9ca7-dbd188ed225e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.016419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 144/295] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:02.024129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f234b-7a5c-4951-b3b1-bb2c47d73494 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.024717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f234b-7a5c-4951-b3b1-bb2c47d73494 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.025205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f234b-7a5c-4951-b3b1-bb2c47d73494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.034788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c788e-82b8-4a2f-8646-5fbc4efa676f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.044640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-aa7f234b-7a5c-4951-b3b1-bb2c47d73494 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:02.046177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-aa7f234b-7a5c-4951-b3b1-bb2c47d73494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:02.073329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.074209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.074491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.074771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.076145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.077651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.079870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 152/296] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:01 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:02.080890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.121992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.125083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 145/297] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:02.126188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f234b-7a5c-4951-b3b1-bb2c47d73494 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.199794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.214025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987cb07-2f96-4356-8f84-fb892130a7ab None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.245218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f234b-7a5c-4951-b3b1-bb2c47d73494 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.248535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.249322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.249598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.249867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.261149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:02.268854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:02.269746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:02.270806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 153/298] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:02.272914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.285266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691685f-d77b-4d9d-aaa6-75d9608901cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.286163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691685f-d77b-4d9d-aaa6-75d9608901cc None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.286968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691685f-d77b-4d9d-aaa6-75d9608901cc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.287238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691685f-d77b-4d9d-aaa6-75d9608901cc None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777/users/e0998e47fd0a492eb19475cdba8f9d30/roles/f4351ca9183346af85c1667531e247fb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.289334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691685f-d77b-4d9d-aaa6-75d9608901cc None admin] RBAC: Authorizing `identity:create_grant(project_id=c109cd55dcce43668065e0c553a39777, user_id=e0998e47fd0a492eb19475cdba8f9d30, role_id=f4351ca9183346af85c1667531e247fb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.290764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691685f-d77b-4d9d-aaa6-75d9608901cc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.303492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 146/299] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:45:02 2022] PUT /identity/v3/projects/c109cd55dcce43668065e0c553a39777/users/e0998e47fd0a492eb19475cdba8f9d30/roles/f4351ca9183346af85c1667531e247fb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:02.304999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1380ef-8b93-47f6-90fc-fead41a7dd18 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.305348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1380ef-8b93-47f6-90fc-fead41a7dd18 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.305625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1380ef-8b93-47f6-90fc-fead41a7dd18 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.306902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 147/300] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:02.307622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1380ef-8b93-47f6-90fc-fead41a7dd18 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.387191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.396181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a3f23-9c8c-41f0-949a-c41992b84002 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.429487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1380ef-8b93-47f6-90fc-fead41a7dd18 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.434089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dab7e-963c-4367-a72c-36c3c08770fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.434916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dab7e-963c-4367-a72c-36c3c08770fb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.435209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dab7e-963c-4367-a72c-36c3c08770fb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.435586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dab7e-963c-4367-a72c-36c3c08770fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.448841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6dab7e-963c-4367-a72c-36c3c08770fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:02.449456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-7d6dab7e-963c-4367-a72c-36c3c08770fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:02.457050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:02.458025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:02.459282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 154/301] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:02.460847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c204-885f-4fc9-9334-f8c8f01bbffc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.460847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c204-885f-4fc9-9334-f8c8f01bbffc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.461010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c204-885f-4fc9-9334-f8c8f01bbffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.476649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.477332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.477611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.477931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.478635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:02.478859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8836c204-885f-4fc9-9334-f8c8f01bbffc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:02.478954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.480079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8836c204-885f-4fc9-9334-f8c8f01bbffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:02.480304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.490715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 148/302] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:02.491562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.560072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 155/303] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:02.560840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836c204-885f-4fc9-9334-f8c8f01bbffc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.613068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01819956-586e-4aab-9093-0fb1828223ec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.651028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.652114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.652972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.653276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.653575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.654463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:02.654779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.656166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.669823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 149/304] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:02.669823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.673539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836c204-885f-4fc9-9334-f8c8f01bbffc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.710869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.711911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.712289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.712689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] PATH_INFO: `/v3/users/0d6bc8ccc5514f6dad6f030230692c0a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.713991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] RBAC: Authorizing `identity:get_user(user_id=0d6bc8ccc5514f6dad6f030230692c0a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.714550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.715607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.717374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 156/305] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/users/0d6bc8ccc5514f6dad6f030230692c0a => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:02.722004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.780037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34ed1-b286-41e8-8149-1deae47302ad None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.818987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.819872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.820225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.820608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.821845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.823276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.829686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 150/306] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:02.829686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.832798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60ea2-429e-468e-9c5d-1d10798609fb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.871035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.871996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.872348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.872705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] PATH_INFO: `/v3/users/0f2f61f4bafa4eada9708700e4350aaa` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.886730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] RBAC: Authorizing `identity:get_user(user_id=0f2f61f4bafa4eada9708700e4350aaa)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.888259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.890222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 157/307] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/users/0f2f61f4bafa4eada9708700e4350aaa => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:02.895335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:02.907573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.934316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962b4e1-02ff-4944-8a19-ac90713cdf96 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:02.948211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:02.979359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:02.980228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:02.980540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:02.980812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] PATH_INFO: `/v3/system/users/0d6bc8ccc5514f6dad6f030230692c0a/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:02.982400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0d6bc8ccc5514f6dad6f030230692c0a, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:02.983774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:02.996283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 151/308] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Jun 7 15:45:02 2022] PUT /identity/v3/system/users/0d6bc8ccc5514f6dad6f030230692c0a/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:03.004598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:03.022575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b0185-48c8-4dea-a32e-e780c358de61 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:03.071641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468e863-4fb6-4b60-8462-b2ecdf6ea407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:03.072506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e863-4fb6-4b60-8462-b2ecdf6ea407 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:03.072862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e863-4fb6-4b60-8462-b2ecdf6ea407 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:03.073319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e863-4fb6-4b60-8462-b2ecdf6ea407 None admin] PATH_INFO: `/v3/groups` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:03.074125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468e863-4fb6-4b60-8462-b2ecdf6ea407 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:03.075697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468e863-4fb6-4b60-8462-b2ecdf6ea407 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:03.093936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 158/309] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Jun 7 15:45:02 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:03.156973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0a590-c79a-416d-8ca1-80de4067166e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:03.199361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528011ca-c78d-4248-bffd-aa6804b2c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:03.199956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528011ca-c78d-4248-bffd-aa6804b2c7ca None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:03.200239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528011ca-c78d-4248-bffd-aa6804b2c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:03.200549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528011ca-c78d-4248-bffd-aa6804b2c7ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:03.201403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528011ca-c78d-4248-bffd-aa6804b2c7ca None admin] RBAC: Adding query filter params (role.id=f4351ca9183346af85c1667531e247fb, user.id=0f2f61f4bafa4eada9708700e4350aaa, scope.system=all) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:03.201720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528011ca-c78d-4248-bffd-aa6804b2c7ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:03.203211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528011ca-c78d-4248-bffd-aa6804b2c7ca None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:03.210041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 152/310] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Jun 7 15:45:02 2022] GET /identity/v3/role_assignments?user.id=0f2f61f4bafa4eada9708700e4350aaa&scope.system=all&role.id=f4351ca9183346af85c1667531e247fb => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:04.589390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043a2f2-3817-4f5d-9a25-249daeee5ff3 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:04.590362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043a2f2-3817-4f5d-9a25-249daeee5ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:04.590874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043a2f2-3817-4f5d-9a25-249daeee5ff3 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:04.592798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 159/311] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:04.597661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ec5f7-91de-4f9c-8d2c-d551edbd4a0e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:04.598045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ec5f7-91de-4f9c-8d2c-d551edbd4a0e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:04.598223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ec5f7-91de-4f9c-8d2c-d551edbd4a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:04.616597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c70ec5f7-91de-4f9c-8d2c-d551edbd4a0e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:04.617780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c70ec5f7-91de-4f9c-8d2c-d551edbd4a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:04.652738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:04.700075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 153/312] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:04.723630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043a2f2-3817-4f5d-9a25-249daeee5ff3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:04.797963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:04.834500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043a2f2-3817-4f5d-9a25-249daeee5ff3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:04.870180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:04.870896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:04.871179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:04.871449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:04.882246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:04.889990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:04.890922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:04.892200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 160/313] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:04.897719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ec5f7-91de-4f9c-8d2c-d551edbd4a0e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.009291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ec5f7-91de-4f9c-8d2c-d551edbd4a0e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:05.051675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:05.053299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.053584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.053858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.054606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:05.054909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:05.056294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:05.065568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 154/314] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:05.071615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.187296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d07e-7eff-4d0f-b26e-f8c777c67ebc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:05.217636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:05.224911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753605bd-96e9-49a3-aa59-0021ab956f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:05.226949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753605bd-96e9-49a3-aa59-0021ab956f5c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.227471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753605bd-96e9-49a3-aa59-0021ab956f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.227952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753605bd-96e9-49a3-aa59-0021ab956f5c None admin] PATH_INFO: `/v3/users/e0998e47fd0a492eb19475cdba8f9d30` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.229481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753605bd-96e9-49a3-aa59-0021ab956f5c None admin] RBAC: Authorizing `identity:get_user(user_id=e0998e47fd0a492eb19475cdba8f9d30)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:05.231193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753605bd-96e9-49a3-aa59-0021ab956f5c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:05.233126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 161/315] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:05 2022] GET /identity/v3/users/e0998e47fd0a492eb19475cdba8f9d30 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:05.244379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.278410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:05.330013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2505c-fdf4-46a4-8b1e-d72545de10ee None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.330339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2505c-fdf4-46a4-8b1e-d72545de10ee None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.330564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2505c-fdf4-46a4-8b1e-d72545de10ee None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.332085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 162/316] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:45:05.338236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb206-192f-435b-9c96-e8475b3df228 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.339593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb206-192f-435b-9c96-e8475b3df228 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.339593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb206-192f-435b-9c96-e8475b3df228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.363034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3464c88-8d8c-46c2-aab2-60cdbf25c1de None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:05.363730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c31bb206-192f-435b-9c96-e8475b3df228 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:05.365345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c31bb206-192f-435b-9c96-e8475b3df228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:05.402726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:05.403592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.403832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.404204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.461981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 163/317] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:05.482653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65700e5f-630e-48a3-9773-20647e5338e6 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.482653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65700e5f-630e-48a3-9773-20647e5338e6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.482653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65700e5f-630e-48a3-9773-20647e5338e6 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.482653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 164/318] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:45:05.487108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59b775-dbb7-4577-b337-606644f284fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.487512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59b775-dbb7-4577-b337-606644f284fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.487792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59b775-dbb7-4577-b337-606644f284fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.493622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:05.495092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:05.505658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0e59b775-dbb7-4577-b337-606644f284fe None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:05.507044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0e59b775-dbb7-4577-b337-606644f284fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:05.526342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:05.581131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 165/319] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:05.585646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59b775-dbb7-4577-b337-606644f284fe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.605415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 155/320] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:05.607065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.651608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:05.694462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59b775-dbb7-4577-b337-606644f284fe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:05.710250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2127d4-ee87-4e27-8690-8c58b8f4cb4c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:05.728024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:05.728846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.729136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.729414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.738213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:05.740292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:05.744798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d910-d148-4ac3-94bc-e0d71fcc5590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:05.745589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d910-d148-4ac3-94bc-e0d71fcc5590 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.745868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d910-d148-4ac3-94bc-e0d71fcc5590 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.746142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d910-d148-4ac3-94bc-e0d71fcc5590 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.747287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04d910-d148-4ac3-94bc-e0d71fcc5590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:05.748185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:05.749086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:05.749937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:05.751032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04d910-d148-4ac3-94bc-e0d71fcc5590 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:05.751099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 166/321] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:05.751162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 156/322] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:05.751486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7b7a-9651-415a-8394-e4dd9910f350 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.751765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7b7a-9651-415a-8394-e4dd9910f350 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.752038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7b7a-9651-415a-8394-e4dd9910f350 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.752593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.753402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 157/323] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:05.757199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7b7a-9651-415a-8394-e4dd9910f350 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.856785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498ba1b-c6a9-498f-a1f0-b0cbf5eb748a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:05.858200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7b7a-9651-415a-8394-e4dd9910f350 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:05.874254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:05.891083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:05.891938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.892075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed6e31-04c6-4f1b-8f3b-f8dbc61d621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:05.892219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.892527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.892673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed6e31-04c6-4f1b-8f3b-f8dbc61d621c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.892961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed6e31-04c6-4f1b-8f3b-f8dbc61d621c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.893338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed6e31-04c6-4f1b-8f3b-f8dbc61d621c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.893736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:05.894070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed6e31-04c6-4f1b-8f3b-f8dbc61d621c None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:05.894375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed6e31-04c6-4f1b-8f3b-f8dbc61d621c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:05.895197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:05.895743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed6e31-04c6-4f1b-8f3b-f8dbc61d621c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:05.897379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 167/324] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:05 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:05.898132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:05.908205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 158/325] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:05.909757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d6582-91a9-4b1a-b16e-ab24bd62bbb5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:05.910052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d6582-91a9-4b1a-b16e-ab24bd62bbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:05.910332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d6582-91a9-4b1a-b16e-ab24bd62bbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:05.927456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ba4d6582-91a9-4b1a-b16e-ab24bd62bbb5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:05.928747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ba4d6582-91a9-4b1a-b16e-ab24bd62bbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:05.986749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:05.999069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 159/326] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:06.002291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd3498-e5cb-4046-ad8d-f0b19f19bc3a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.003344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d6582-91a9-4b1a-b16e-ab24bd62bbb5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:06.037729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.038397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.038675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.038950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] PATH_INFO: `/v3/groups` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.039632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:06.041084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:06.054174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 168/327] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Jun 7 15:45:05 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:06.054997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:06.113343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d6582-91a9-4b1a-b16e-ab24bd62bbb5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.118692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:06.163394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.164614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.165083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.165361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.166217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] RBAC: Adding query filter params (role.id=f4351ca9183346af85c1667531e247fb, scope.project.id=c109cd55dcce43668065e0c553a39777, user.id=e0998e47fd0a492eb19475cdba8f9d30) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:06.166528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:06.168012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:06.174747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 160/328] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:05 2022] GET /identity/v3/role_assignments?user.id=e0998e47fd0a492eb19475cdba8f9d30&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=f4351ca9183346af85c1667531e247fb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:06.175673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:06.202936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658307aa-bec4-4486-80cc-3eb23bb27d3a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.243897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:06.265715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.266611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.266948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.267224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] PATH_INFO: `/v3/users/0d6bc8ccc5514f6dad6f030230692c0a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.268572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] RBAC: Authorizing `identity:get_user(user_id=0d6bc8ccc5514f6dad6f030230692c0a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:06.270093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:06.273794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 169/329] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:06 2022] GET /identity/v3/users/0d6bc8ccc5514f6dad6f030230692c0a => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:06.281060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:06.309045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29dbd0-f1d4-42b0-9a66-257fac714b80 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.352051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.352917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.353201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.353473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.370354 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:06.379035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:06.380054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:06.381360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:06.381961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 161/330] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:06.386441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:06.430452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22426ee6-b9b8-49ee-aefa-9bf827e1ae26 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.448280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:06.478682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.479344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.479637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.479909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.480777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] RBAC: Adding query filter params (role.id=98ebd7033f624d02bef09757068ee356, user.id=0d6bc8ccc5514f6dad6f030230692c0a, scope.system=all) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:06.481093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:06.482545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:06.499690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 170/331] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Jun 7 15:45:06 2022] GET /identity/v3/role_assignments?user.id=0d6bc8ccc5514f6dad6f030230692c0a&scope.system=all&role.id=98ebd7033f624d02bef09757068ee356 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:06.525745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72a217-ad6b-4943-9fa2-5234a28fe6fc None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.542823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:06.567903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.568990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.569495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.569969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.570973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:06.571488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:06.573164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:06.584760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 162/332] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:06.590058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:06.711216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac9ba1-386f-4317-9aa7-d0f3c1cf60c6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.748524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a4476-0871-4db1-a451-1b545b70dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.749405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a4476-0871-4db1-a451-1b545b70dbdb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.749686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a4476-0871-4db1-a451-1b545b70dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.750013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a4476-0871-4db1-a451-1b545b70dbdb None admin] PATH_INFO: `/v3/users/0f2f61f4bafa4eada9708700e4350aaa` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.751176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a4476-0871-4db1-a451-1b545b70dbdb None admin] RBAC: Authorizing `identity:get_user(user_id=0f2f61f4bafa4eada9708700e4350aaa)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:06.752665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a4476-0871-4db1-a451-1b545b70dbdb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:06.754613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 171/333] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:06 2022] GET /identity/v3/users/0f2f61f4bafa4eada9708700e4350aaa => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:06.760872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:06.879736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad3cdd-8657-49d8-b324-0c8a1b020b04 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:06.917333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0fdab-3edb-4780-a1b2-17a3c66815dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:06.918207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fdab-3edb-4780-a1b2-17a3c66815dd None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:06.918490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fdab-3edb-4780-a1b2-17a3c66815dd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:06.918772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fdab-3edb-4780-a1b2-17a3c66815dd None admin] PATH_INFO: `/v3/system/users/0f2f61f4bafa4eada9708700e4350aaa/roles/f4351ca9183346af85c1667531e247fb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:06.920314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0fdab-3edb-4780-a1b2-17a3c66815dd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f2f61f4bafa4eada9708700e4350aaa, role_id=f4351ca9183346af85c1667531e247fb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:06.921777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0fdab-3edb-4780-a1b2-17a3c66815dd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:06.933549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 163/334] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Jun 7 15:45:06 2022] PUT /identity/v3/system/users/0f2f61f4bafa4eada9708700e4350aaa/roles/f4351ca9183346af85c1667531e247fb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:09.192769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20825ff1-f9c9-43ff-8a29-ada017ee7b3c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.193892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20825ff1-f9c9-43ff-8a29-ada017ee7b3c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.194372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20825ff1-f9c9-43ff-8a29-ada017ee7b3c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.196003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 172/335] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:09.201895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0878-253e-413f-ac3f-04e71f9f7edf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.202824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0878-253e-413f-ac3f-04e71f9f7edf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.203308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd0878-253e-413f-ac3f-04e71f9f7edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.227995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3cbd0878-253e-413f-ac3f-04e71f9f7edf None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:09.229570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3cbd0878-253e-413f-ac3f-04e71f9f7edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:09.266391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:09.306302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 164/336] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:09.343001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20825ff1-f9c9-43ff-8a29-ada017ee7b3c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:09.398683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:09.459013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac61f5-47d9-4755-9e4f-f0ad5397b4c5 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.459193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac61f5-47d9-4755-9e4f-f0ad5397b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.459512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac61f5-47d9-4755-9e4f-f0ad5397b4c5 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.460846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 165/337] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:45:09.471151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20825ff1-f9c9-43ff-8a29-ada017ee7b3c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:09.473370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188c34c-85b9-41ed-b8c6-b4a6e1ee76ad None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.474733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188c34c-85b9-41ed-b8c6-b4a6e1ee76ad None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.475221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188c34c-85b9-41ed-b8c6-b4a6e1ee76ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.521949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2188c34c-85b9-41ed-b8c6-b4a6e1ee76ad None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:09.523478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2188c34c-85b9-41ed-b8c6-b4a6e1ee76ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:09.525051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:09.525862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.526169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.526420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.551533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:09.559564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:09.560628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:09.561455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:09.564382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 173/338] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:09.571440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:09.674603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 166/339] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:09.691538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04ab98-9b64-4475-9223-8bf2a4f4233d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:09.694931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:09.704732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188c34c-85b9-41ed-b8c6-b4a6e1ee76ad None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:09.773190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:09.774110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.774535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.774958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.775875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:09.776331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:09.777885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:09.795948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 174/340] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:09.801938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:09.837362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:09.841115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188c34c-85b9-41ed-b8c6-b4a6e1ee76ad None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:09.892596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26bc18-1342-4637-903e-b983b500dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:09.894757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26bc18-1342-4637-903e-b983b500dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.895041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26bc18-1342-4637-903e-b983b500dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.895320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26bc18-1342-4637-903e-b983b500dfe4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.919358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26bc18-1342-4637-903e-b983b500dfe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bf26bc18-1342-4637-903e-b983b500dfe4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:09.931373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:09.932549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:09.933338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 167/341] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:09.933989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a5a18-7435-4baf-9b76-c3e92e69d036 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:09.934277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a5a18-7435-4baf-9b76-c3e92e69d036 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:09.936447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a5a18-7435-4baf-9b76-c3e92e69d036 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:09.940894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 168/342] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:09.941662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a5a18-7435-4baf-9b76-c3e92e69d036 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:10.052647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aae782-e3fd-4340-a796-3352329b68c0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:10.124047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:10.125091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.125604 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.126075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.157801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a5a18-7435-4baf-9b76-c3e92e69d036 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:10.190073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:10.234005 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e54704-d74c-41d7-b53c-1cae2aeb0c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:10.234676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e54704-d74c-41d7-b53c-1cae2aeb0c96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.234962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e54704-d74c-41d7-b53c-1cae2aeb0c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.235238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e54704-d74c-41d7-b53c-1cae2aeb0c96 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.235991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e54704-d74c-41d7-b53c-1cae2aeb0c96 None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:10.236300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e54704-d74c-41d7-b53c-1cae2aeb0c96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:10.237753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e54704-d74c-41d7-b53c-1cae2aeb0c96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:10.240143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:10.241677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:10.256782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 169/343] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:10.258194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613765-4e8f-4f34-98e5-e8ba3fcef2aa None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.258484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613765-4e8f-4f34-98e5-e8ba3fcef2aa None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.258759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613765-4e8f-4f34-98e5-e8ba3fcef2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.286533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-59613765-4e8f-4f34-98e5-e8ba3fcef2aa None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:10.288103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-59613765-4e8f-4f34-98e5-e8ba3fcef2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:10.328459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:10.369837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 175/344] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:10.375591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:10.408387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 170/345] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:10.408387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9f69-44f7-4605-b536-4a9a99ccef14 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.408387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9f69-44f7-4605-b536-4a9a99ccef14 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.408387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9f69-44f7-4605-b536-4a9a99ccef14 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.409432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 171/346] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:10.410270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9f69-44f7-4605-b536-4a9a99ccef14 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:10.456869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:10.513444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17bd8c-75f8-48d1-b543-49d33dded6e1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:10.554313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9f69-44f7-4605-b536-4a9a99ccef14 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:10.558929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe405-6c5a-4d9c-835d-264bff5aa018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:10.558929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe405-6c5a-4d9c-835d-264bff5aa018 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.558929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe405-6c5a-4d9c-835d-264bff5aa018 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.558929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe405-6c5a-4d9c-835d-264bff5aa018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.604335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702b35b-5665-45ba-b9d9-ba8fffae6ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:10.605351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b35b-5665-45ba-b9d9-ba8fffae6ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.606481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b35b-5665-45ba-b9d9-ba8fffae6ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.606813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b35b-5665-45ba-b9d9-ba8fffae6ab7 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.609458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702b35b-5665-45ba-b9d9-ba8fffae6ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:10.609588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702b35b-5665-45ba-b9d9-ba8fffae6ab7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:10.611475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 172/347] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:10 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:10.613057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66408cf-5fe8-46b6-a7ca-5a90db267b56 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.613057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66408cf-5fe8-46b6-a7ca-5a90db267b56 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.613252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66408cf-5fe8-46b6-a7ca-5a90db267b56 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.614488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 173/348] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:10.615557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32da72-8aa9-4ec4-b520-9c83de3c0c39 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.615842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32da72-8aa9-4ec4-b520-9c83de3c0c39 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.616117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32da72-8aa9-4ec4-b520-9c83de3c0c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.637717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-db32da72-8aa9-4ec4-b520-9c83de3c0c39 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:10.639507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-db32da72-8aa9-4ec4-b520-9c83de3c0c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:10.649620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfe405-6c5a-4d9c-835d-264bff5aa018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:10.651101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfe405-6c5a-4d9c-835d-264bff5aa018 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:10.657059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:10.716770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 174/349] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:10.717922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32da72-8aa9-4ec4-b520-9c83de3c0c39 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:10.723363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:10.773428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 176/350] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:10.775037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39833691-d09a-4e7a-81eb-1222ca194773 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.775322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39833691-d09a-4e7a-81eb-1222ca194773 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.775594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39833691-d09a-4e7a-81eb-1222ca194773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.802808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-39833691-d09a-4e7a-81eb-1222ca194773 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:10.804188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-39833691-d09a-4e7a-81eb-1222ca194773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:10.859285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32da72-8aa9-4ec4-b520-9c83de3c0c39 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:10.895883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 177/351] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:10.896890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39833691-d09a-4e7a-81eb-1222ca194773 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:10.900577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:10.901653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:10.902140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:10.902618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:10.915073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:10.925870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:10.927417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:10.928231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:10.929113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 175/352] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:45:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:10.930340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:10.945748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.007795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.019407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39833691-d09a-4e7a-81eb-1222ca194773 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.056047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2376-0917-4978-a728-a548e5909b06 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.059067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.059934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.060218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.060518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.070561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.072052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:11.074762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 178/353] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:10 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:11.074921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.102289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.103393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.103393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.103617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.114904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:11.122858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:11.123764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:11.124644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 176/354] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:11.125317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.200958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b1e64-3a08-4db8-a9ab-f5dcb140e351 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.208756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.242862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.243720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.244241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.244565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.245770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.246846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4514d7-eb8e-4db6-a191-63b731a94fa0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.247234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:11.248924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 179/355] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:11.249684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.279947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.289041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.290098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.290383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.290651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.304228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:11.312427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:11.313490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 177/356] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:11.319192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.377613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642dd41-4688-46db-9fa1-29270dcb7ca1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.414055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.414811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.414986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.415259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.415999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:11.416299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.417689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:11.426506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 180/357] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:11.428180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.446530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c44f4-97e5-4c88-8c60-2120dad47c14 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.481997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.488085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f975096-3e00-4844-9ea5-1602bfe6a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.488991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f975096-3e00-4844-9ea5-1602bfe6a214 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.490228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f975096-3e00-4844-9ea5-1602bfe6a214 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.490708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f975096-3e00-4844-9ea5-1602bfe6a214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.491641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f975096-3e00-4844-9ea5-1602bfe6a214 None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=c109cd55dcce43668065e0c553a39777) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:11.491953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f975096-3e00-4844-9ea5-1602bfe6a214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.493860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f975096-3e00-4844-9ea5-1602bfe6a214 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:11.501603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 178/358] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=98ebd7033f624d02bef09757068ee356 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:11.502878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26fbd2-2c2a-4d2e-ab68-deeef3f37b0c None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.503158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26fbd2-2c2a-4d2e-ab68-deeef3f37b0c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.503432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26fbd2-2c2a-4d2e-ab68-deeef3f37b0c None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.505714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 179/359] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:11.506990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26fbd2-2c2a-4d2e-ab68-deeef3f37b0c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.546844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.567581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126568aa-3e1e-49a9-82a4-b82bfa2208eb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.611855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.612567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.612847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.613125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.613876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:11.614767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.617123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:11.626411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 181/360] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:11.632259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.654737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26fbd2-2c2a-4d2e-ab68-deeef3f37b0c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.713313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.714189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.714506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.714766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.715927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.717405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:11.720058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 180/361] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:11.720797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.770108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721287d-be55-4909-8db8-66aa8121d3d4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.801476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.813333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac738007-7d70-464d-83b5-5dadb06a49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.813333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac738007-7d70-464d-83b5-5dadb06a49fc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.813333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac738007-7d70-464d-83b5-5dadb06a49fc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.813680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac738007-7d70-464d-83b5-5dadb06a49fc None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.814360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac738007-7d70-464d-83b5-5dadb06a49fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:11.814661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac738007-7d70-464d-83b5-5dadb06a49fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.816028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac738007-7d70-464d-83b5-5dadb06a49fc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:11.832777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 182/362] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:11.832777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be49fb5-2908-4fc2-b18f-30721d728e7d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.832777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be49fb5-2908-4fc2-b18f-30721d728e7d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.832777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be49fb5-2908-4fc2-b18f-30721d728e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.844523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0036f-ad81-46ea-835e-7eb740ca7f4a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:11.850646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7be49fb5-2908-4fc2-b18f-30721d728e7d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:11.851837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7be49fb5-2908-4fc2-b18f-30721d728e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:11.855234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:11.883965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:11.884848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:11.885136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:11.885418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:11.932829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 183/363] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:11.933567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be49fb5-2908-4fc2-b18f-30721d728e7d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:11.971574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:11.973069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:12.043094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:12.072969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be49fb5-2908-4fc2-b18f-30721d728e7d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.098039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 181/364] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:12.098907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.117220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:12.118107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:12.118382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:12.118653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] PATH_INFO: `/v3/users/0f2f61f4bafa4eada9708700e4350aaa` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:12.119854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] RBAC: Authorizing `identity:get_user(user_id=0f2f61f4bafa4eada9708700e4350aaa)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:12.121789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:12.123716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 184/365] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:45:11 2022] GET /identity/v3/users/0f2f61f4bafa4eada9708700e4350aaa => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:12.124519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.142847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:12.234273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19241dd6-d0e7-4457-9551-c82cfe0a8a83 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.266819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ff02b-29b6-4943-ad51-053893c86e6c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.286455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:12.288326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:12.288900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:12.289471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:12.290761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:12.291658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:12.293352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:12.300743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 182/366] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=98ebd7033f624d02bef09757068ee356 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:12.302333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.311388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:12.312067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:12.312349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:12.312687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:12.337275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:12.400139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:12.401761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:12.413717 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:12.421718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8434-87a3-4091-bf1f-1e002bbc2813 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.460024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:12.461115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:12.461554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:12.461964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:12.475095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:12.483329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:12.484258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:12.485156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:12.487476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 183/367] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:12.488521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.531469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 185/368] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:12.532745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.588637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:12.601239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a19bbd-13eb-4bcf-a447-bc01cada248a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.636249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:12.637395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:12.637829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:12.638240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:12.639544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:12.641147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:12.643297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 184/369] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:12.644214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.649100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc42d-8487-4f3b-a98b-2b4b534c896b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.659182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:12.686534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:12.687987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:12.687987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:12.687987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:12.688941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] RBAC: Adding query filter params (role.id=f4351ca9183346af85c1667531e247fb, user.id=0f2f61f4bafa4eada9708700e4350aaa, scope.system=all) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:12.689336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:12.690895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:12.704983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 186/370] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/role_assignments?user.id=0f2f61f4bafa4eada9708700e4350aaa&scope.system=all&role.id=f4351ca9183346af85c1667531e247fb => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:12.705960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.774070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da03a0b-37d7-4f71-b8f7-ebab575c5d45 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.814886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:12.817102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:12.818273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:12.818702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:12.819672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:12.820169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:12.821756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:12.832854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 185/371] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:12.834056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:12.859666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:12.964716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886934-7476-4454-82e5-d3bb84087ef1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:12.969491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751754d-459d-47db-9c20-94677bd42aed None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:13.013821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.014331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.014937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.015368 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.015686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.015804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.016197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.017775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.019099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.019386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.021482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.022016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.023494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 187/372] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:13.023905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 186/373] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:12 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:13.024593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:13.029191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:13.069885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:13.126499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:13.157013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586759c5-39da-47b7-9324-050574a477c7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:13.164545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93e86-c622-40c9-ae17-57f7ccad7c87 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:13.198040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.199098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.199480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.199857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.201703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.203270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.205481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 187/374] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:13 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:13.206441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:13.210681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.211541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.211969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.212390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.305029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.307900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.339592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a40308-a1ad-423f-b5d7-0c76b2d1795b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:13.352089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:13.386816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.387956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.388423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.389246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.390501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:13.390963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.392806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.400881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 188/375] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:13 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:13.401897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:13.428380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:13.444528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 188/376] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:13.446957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:13.557530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae5ddf-1e04-4e6b-9bf9-ae0b8ca7c1ec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:13.574627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc960a-742c-467a-a2f4-89dee8d47221 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:13.606668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.607649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.608057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.609158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.610968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=c109cd55dcce43668065e0c553a39777) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:13.611479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.613652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.623547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 189/377] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:13 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:13.629736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.631115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.631749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.632267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] PATH_INFO: `/v3/projects/3e30cc2a9bc04d579b04c2060b966f87` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.634151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] RBAC: Authorizing `identity:get_project(project_id=3e30cc2a9bc04d579b04c2060b966f87)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.636183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.643272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 189/378] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:13 2022] GET /identity/v3/projects/3e30cc2a9bc04d579b04c2060b966f87 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:13.645163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:13.750586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:13.779728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e7daf-b527-4f6a-85ad-a91beef4556d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:13.823784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb8c0a-7a49-4e75-928d-98a16c454132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:13.825148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8c0a-7a49-4e75-928d-98a16c454132 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:13.825773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8c0a-7a49-4e75-928d-98a16c454132 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:13.826261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8c0a-7a49-4e75-928d-98a16c454132 None admin] PATH_INFO: `/v3/groups/44194820779141b5aa6b2cdcc4fbac78` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:13.836734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb8c0a-7a49-4e75-928d-98a16c454132 None admin] RBAC: Authorizing `identity:get_group(group_id=44194820779141b5aa6b2cdcc4fbac78)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:13.838845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb8c0a-7a49-4e75-928d-98a16c454132 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:13.842172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 190/379] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:13 2022] GET /identity/v3/groups/44194820779141b5aa6b2cdcc4fbac78 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:13.847566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:13.851539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:13.969904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c938dd7-b7c0-49bd-bb45-e1ce4bc36695 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:14.014129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a0f2e-c01f-4e9c-a104-a47386f98fea None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:14.014450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a0f2e-c01f-4e9c-a104-a47386f98fea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:14.014693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a0f2e-c01f-4e9c-a104-a47386f98fea None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:14.016203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 191/380] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:14.021531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd61a2c-187a-4f70-a292-76a2e650ff5f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:14.022116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd61a2c-187a-4f70-a292-76a2e650ff5f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:14.022387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:14.022823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd61a2c-187a-4f70-a292-76a2e650ff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:14.023049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:14.023337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:14.023614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:14.024521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] RBAC: Adding query filter params (group.id=44194820779141b5aa6b2cdcc4fbac78, role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=3e30cc2a9bc04d579b04c2060b966f87) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:14.024818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:14.026261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:14.033421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 190/381] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:13 2022] GET /identity/v3/role_assignments?group.id=44194820779141b5aa6b2cdcc4fbac78&scope.project.id=3e30cc2a9bc04d579b04c2060b966f87&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:14.068433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0bd61a2c-187a-4f70-a292-76a2e650ff5f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:14.069894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0bd61a2c-187a-4f70-a292-76a2e650ff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:14.180078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 192/382] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 15:45:14.215591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:14.304298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:14.346302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60baceb7-2c22-4d4e-85d5-8ed09d6ee470 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:14.390627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:14.391824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:14.392305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:14.392823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:14.406568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:14.414398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:14.415269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:14.416498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 191/383] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:14.421397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd61a2c-187a-4f70-a292-76a2e650ff5f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:14.598268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:14.632428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd61a2c-187a-4f70-a292-76a2e650ff5f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:14.683051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:14.683641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:14.683925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:14.684202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:14.684983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:14.685290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:14.686676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:14.700280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 193/384] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:14.705920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:14.816654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518be7c-319e-4687-a328-0814e53a66e6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:14.844257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:14.856541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:14.857399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:14.857683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:14.857959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:14.865422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:14.866871 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:14.868532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 192/385] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:14 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:14.874023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:15.003582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2aa10-4c6a-449a-b5b9-c636194e8531 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:15.040012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bdfd5-7c0e-4880-9541-56083035648b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:15.040900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bdfd5-7c0e-4880-9541-56083035648b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:15.041238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bdfd5-7c0e-4880-9541-56083035648b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:15.041446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bdfd5-7c0e-4880-9541-56083035648b None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:15.042636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bdfd5-7c0e-4880-9541-56083035648b None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:15.044072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bdfd5-7c0e-4880-9541-56083035648b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:15.046087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 194/386] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:14 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:15.057529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:15.202926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a18674-cf1e-4fbd-b232-98a4a9135b1b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:15.244000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d43d9-41c9-484b-aa1b-7a75f5974d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:15.244923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d43d9-41c9-484b-aa1b-7a75f5974d53 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:15.245205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d43d9-41c9-484b-aa1b-7a75f5974d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:15.245482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d43d9-41c9-484b-aa1b-7a75f5974d53 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777/groups/847db62643e9457fbef40769ec290138/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:15.247471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d43d9-41c9-484b-aa1b-7a75f5974d53 None admin] RBAC: Authorizing `identity:create_grant(project_id=c109cd55dcce43668065e0c553a39777, group_id=847db62643e9457fbef40769ec290138, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:15.248952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d43d9-41c9-484b-aa1b-7a75f5974d53 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:15.271759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 193/387] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Jun 7 15:45:15 2022] PUT /identity/v3/projects/c109cd55dcce43668065e0c553a39777/groups/847db62643e9457fbef40769ec290138/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:15.423706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4b4c-9790-4877-b2be-883c0e6644e2 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:15.424769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4b4c-9790-4877-b2be-883c0e6644e2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:15.425250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4b4c-9790-4877-b2be-883c0e6644e2 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:15.427681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 195/388] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:15.433765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2412d-7afc-4267-900f-87a64e573a14 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:15.434652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2412d-7afc-4267-900f-87a64e573a14 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:15.435122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2412d-7afc-4267-900f-87a64e573a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:15.453538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7ab2412d-7afc-4267-900f-87a64e573a14 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:15.455304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7ab2412d-7afc-4267-900f-87a64e573a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:15.537565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:15.570583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 194/389] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:15.602796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e4b4c-9790-4877-b2be-883c0e6644e2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:15.607090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:15.742798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e4b4c-9790-4877-b2be-883c0e6644e2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:15.799940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:15.801140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:15.801657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:15.802457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:15.819811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:15.829076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:15.829948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:15.830723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:15.840849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 196/390] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:15.846997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2412d-7afc-4267-900f-87a64e573a14 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:16.031234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2412d-7afc-4267-900f-87a64e573a14 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:16.086205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99234a-5960-4a31-90f2-ae72c3986a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:16.086797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99234a-5960-4a31-90f2-ae72c3986a0e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.087071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99234a-5960-4a31-90f2-ae72c3986a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.087334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99234a-5960-4a31-90f2-ae72c3986a0e None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.088092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99234a-5960-4a31-90f2-ae72c3986a0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:16.088411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99234a-5960-4a31-90f2-ae72c3986a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:16.089809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99234a-5960-4a31-90f2-ae72c3986a0e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:16.105171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 195/391] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:16.111643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:16.248806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:16.278559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7a1f-8b5f-44a1-8f85-8734c2815a7e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.278996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7a1f-8b5f-44a1-8f85-8734c2815a7e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.279370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7a1f-8b5f-44a1-8f85-8734c2815a7e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.281187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805714c6-f21b-41ca-bdcb-b85e57c6b762 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:16.282098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 196/392] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:16.289193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679ca19-3653-4c88-963a-c7f4bf0c042e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.289593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679ca19-3653-4c88-963a-c7f4bf0c042e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.289969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679ca19-3653-4c88-963a-c7f4bf0c042e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.321455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a679ca19-3653-4c88-963a-c7f4bf0c042e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:16.322817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a679ca19-3653-4c88-963a-c7f4bf0c042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:16.331637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:16.333422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.334025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.334560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.335972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:16.337674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:16.339816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 197/393] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:16 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:16.348742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:16.432748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:16.475736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 197/394] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:16.502207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7374e3-86e5-4a8f-ba10-8b07380f9a7f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:16.511465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679ca19-3653-4c88-963a-c7f4bf0c042e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:16.535199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:16.540211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:16.542073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.542742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.543389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.545392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:16.547208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:16.549230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 198/395] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:16 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:16.558692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:16.683602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679ca19-3653-4c88-963a-c7f4bf0c042e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:16.706540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47501efe-6e30-4601-b995-9f770d41a9d0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:16.728951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fed968-adeb-4b7f-b37c-fcba720d9b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:16.729968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed968-adeb-4b7f-b37c-fcba720d9b7b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.730297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed968-adeb-4b7f-b37c-fcba720d9b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.730959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed968-adeb-4b7f-b37c-fcba720d9b7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.744299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9a617-8452-4821-a5a5-3ba94af0ce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:16.745495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9a617-8452-4821-a5a5-3ba94af0ce75 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.745890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9a617-8452-4821-a5a5-3ba94af0ce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.746262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9a617-8452-4821-a5a5-3ba94af0ce75 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/groups/847db62643e9457fbef40769ec290138/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.747166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fed968-adeb-4b7f-b37c-fcba720d9b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:16.748816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9a617-8452-4821-a5a5-3ba94af0ce75 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f41ab15737f42cd803bf7f27bc66b36, group_id=847db62643e9457fbef40769ec290138, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:16.750396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9a617-8452-4821-a5a5-3ba94af0ce75 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b1fed968-adeb-4b7f-b37c-fcba720d9b7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:16.757457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:16.758761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:16.759567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 198/396] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:45:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:16.760205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd1da-9973-4ed8-b464-74c3ed19e690 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.761149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd1da-9973-4ed8-b464-74c3ed19e690 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.761149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd1da-9973-4ed8-b464-74c3ed19e690 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.764404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 199/397] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:16.766601 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddd1da-9973-4ed8-b464-74c3ed19e690 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:16.774361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 199/398] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Jun 7 15:45:16 2022] PUT /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/groups/847db62643e9457fbef40769ec290138/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:16.782279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c720a-7faa-443e-9a57-106c719818a4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.782279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c720a-7faa-443e-9a57-106c719818a4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.782732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c720a-7faa-443e-9a57-106c719818a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.813481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-018c720a-7faa-443e-9a57-106c719818a4 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:16.813867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:16.815473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-018c720a-7faa-443e-9a57-106c719818a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:16.904916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddd1da-9973-4ed8-b464-74c3ed19e690 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:16.951438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:16.952424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:16.953001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:16.953447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:16.954475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:16.955461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:16.955927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:16.958881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:16.971673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 200/399] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:16.978343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:16.991635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 200/400] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:17.024969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c720a-7faa-443e-9a57-106c719818a4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.094685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:17.100599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dba4c-ae85-48a5-9fde-cf2c6f0705d8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.137195 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef4080-e345-46b1-b2c7-6ca8affc753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.138341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef4080-e345-46b1-b2c7-6ca8affc753c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.138793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef4080-e345-46b1-b2c7-6ca8affc753c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.139198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef4080-e345-46b1-b2c7-6ca8affc753c None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.140556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef4080-e345-46b1-b2c7-6ca8affc753c None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:17.142144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef4080-e345-46b1-b2c7-6ca8affc753c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:17.144463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 201/401] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:16 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:17.146118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fadccb-ef11-4fc8-885e-44334fb57c28 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.146547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fadccb-ef11-4fc8-885e-44334fb57c28 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.146962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fadccb-ef11-4fc8-885e-44334fb57c28 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.148756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 202/402] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:17.149994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fadccb-ef11-4fc8-885e-44334fb57c28 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.179622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c720a-7faa-443e-9a57-106c719818a4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.230581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989fd50-e1ef-4f17-a9e7-4f50a0b37a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.233265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989fd50-e1ef-4f17-a9e7-4f50a0b37a51 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.233710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989fd50-e1ef-4f17-a9e7-4f50a0b37a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.234137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989fd50-e1ef-4f17-a9e7-4f50a0b37a51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.252456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989fd50-e1ef-4f17-a9e7-4f50a0b37a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5989fd50-e1ef-4f17-a9e7-4f50a0b37a51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:17.261695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:17.262768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:17.263719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:17.266168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 201/403] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:17.267686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5d656-6712-4987-9af7-5bc377b220d0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.268117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5d656-6712-4987-9af7-5bc377b220d0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.268558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5d656-6712-4987-9af7-5bc377b220d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.273367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fadccb-ef11-4fc8-885e-44334fb57c28 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.290363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c0d5d656-6712-4987-9af7-5bc377b220d0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:17.292060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c0d5d656-6712-4987-9af7-5bc377b220d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:17.319066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.320296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.320674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.321020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.322996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:17.324788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:17.327873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 203/404] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:17.328835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.345459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:17.373816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 202/405] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:17.375236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:17.375901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5d656-6712-4987-9af7-5bc377b220d0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.452673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1122519-9836-47e0-bb69-f724dc454d73 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.491883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.493355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.493853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.494324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.495324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:17.495823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:17.497469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:17.506006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 204/406] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:17.507334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.516941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5d656-6712-4987-9af7-5bc377b220d0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.554813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.556092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.556634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.557404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/groups/847db62643e9457fbef40769ec290138/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.560264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] RBAC: Authorizing `identity:create_grant(project_id=2f41ab15737f42cd803bf7f27bc66b36, group_id=847db62643e9457fbef40769ec290138, role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:17.562030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:17.585146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 203/407] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Jun 7 15:45:17 2022] PUT /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36/groups/847db62643e9457fbef40769ec290138/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:17.586965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.619338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:17.625326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3703e76-62ac-43c3-8c6c-3738967a0f18 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.646267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:17.681161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.682369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.682369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.682537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.697830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:17.714112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:17.715100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:17.716602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 205/408] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:17.721320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.727347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f21649-2ec8-4b15-9921-310957298a2b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.769505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.770868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.771347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.771886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.773638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:17.775427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:17.778038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 204/409] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:17.784384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.857845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22cd8b-50a6-46a3-8d1a-47b968d2384d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.900591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:17.901504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:17.901956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:17.902367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:17.903252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:17.903704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:17.905241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:17.916989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 206/410] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:17.922653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:17.958906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:17.986553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9623e1f-7206-400b-af43-2c2677202d13 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:17.998646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:18.030747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:18.031824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.032110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.034246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.036808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:18.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:18.041071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 205/411] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:18.052387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:18.061619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf05b8-2eef-4e8a-8fce-e0b7cc801663 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:18.141169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:18.142040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.142316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.142585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] PATH_INFO: `/v3/groups/44194820779141b5aa6b2cdcc4fbac78` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.147459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] RBAC: Authorizing `identity:get_group(group_id=44194820779141b5aa6b2cdcc4fbac78)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:18.148949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:18.150603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 207/412] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:17 2022] GET /identity/v3/groups/44194820779141b5aa6b2cdcc4fbac78 => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:18.158780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:18.167987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfe28c-bff4-4156-be41-86c19a551266 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:18.205926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629b0ca-6268-4980-bb4c-9ff8184a155b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:18.206813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629b0ca-6268-4980-bb4c-9ff8184a155b None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.207094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629b0ca-6268-4980-bb4c-9ff8184a155b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.207369 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629b0ca-6268-4980-bb4c-9ff8184a155b None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777/groups/847db62643e9457fbef40769ec290138/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.210356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629b0ca-6268-4980-bb4c-9ff8184a155b None admin] RBAC: Authorizing `identity:create_grant(project_id=c109cd55dcce43668065e0c553a39777, group_id=847db62643e9457fbef40769ec290138, role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:18.211807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629b0ca-6268-4980-bb4c-9ff8184a155b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:18.229016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 206/413] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Jun 7 15:45:18 2022] PUT /identity/v3/projects/c109cd55dcce43668065e0c553a39777/groups/847db62643e9457fbef40769ec290138/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:18.302837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a63e3-11f8-4786-8dc9-5cbde10774e5 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.303037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a63e3-11f8-4786-8dc9-5cbde10774e5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.303317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a63e3-11f8-4786-8dc9-5cbde10774e5 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.304954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 207/414] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:18.324034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3902-b754-450c-b8b7-25ce6339c378 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.324403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3902-b754-450c-b8b7-25ce6339c378 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.324613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3902-b754-450c-b8b7-25ce6339c378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.340973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:18.357729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f7dd9-fcae-4869-8128-8fd4ce105111 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:18.358932 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c8dc3902-b754-450c-b8b7-25ce6339c378 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:18.360201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c8dc3902-b754-450c-b8b7-25ce6339c378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:18.406334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:18.407222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.407506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.407783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] PATH_INFO: `/v3/projects/3e30cc2a9bc04d579b04c2060b966f87` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.412478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] RBAC: Authorizing `identity:get_project(project_id=3e30cc2a9bc04d579b04c2060b966f87)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:18.415224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:18.417293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 208/415] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:18 2022] GET /identity/v3/projects/3e30cc2a9bc04d579b04c2060b966f87 => generated 344 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:18.428735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:18.428927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:18.517074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 208/416] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:18.543288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc3902-b754-450c-b8b7-25ce6339c378 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:18.587394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c1ec-1c04-42cb-88c8-77d6e3ffc364 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:18.661846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc3902-b754-450c-b8b7-25ce6339c378 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:18.662433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:18.663326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.663609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.663887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] PATH_INFO: `/v3/projects/3e30cc2a9bc04d579b04c2060b966f87/groups/44194820779141b5aa6b2cdcc4fbac78/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.665951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e30cc2a9bc04d579b04c2060b966f87, group_id=44194820779141b5aa6b2cdcc4fbac78, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:18.667408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:18.701988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:18.702785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.703080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.703415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.707543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 209/417] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Jun 7 15:45:18 2022] PUT /identity/v3/projects/3e30cc2a9bc04d579b04c2060b966f87/groups/44194820779141b5aa6b2cdcc4fbac78/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:18.727372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:18.735449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:18.737617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:18.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 209/418] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:18.743015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:18.777774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:18.901070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d933b8a-0f22-4c52-944a-6454ec325173 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:18.937248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:18.937935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:18.938266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:18.938549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:18.939314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:18.939622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:18.941048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:18.950603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 210/419] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:18.956160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:19.000373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:19.084376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb0fab-8234-4d03-9aaa-c6f717b5703a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:19.127018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:19.127682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:19.127960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:19.128232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:19.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:19.221953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:19.252161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:19.357089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 210/420] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:19.362579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:19.455285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:19.487727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42163a5d-aa6b-446c-9fb0-f47d3cacbd3e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:19.530069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:19.531033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:19.531316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:19.531593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:19.532811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:19.534248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:19.536916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 211/421] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:19 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:19.542292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:19.648525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9d176-3553-45bc-816b-bb819f968782 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:19.679305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:19.688180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bdbd5-4e07-4708-9f26-e98a1ca52179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:19.688955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bdbd5-4e07-4708-9f26-e98a1ca52179 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:19.689379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bdbd5-4e07-4708-9f26-e98a1ca52179 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:19.689665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bdbd5-4e07-4708-9f26-e98a1ca52179 None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:19.690841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bdbd5-4e07-4708-9f26-e98a1ca52179 None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:19.692573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bdbd5-4e07-4708-9f26-e98a1ca52179 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:19.696694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 211/422] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:19 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:19.700478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:19.726338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69df1c-3d59-4337-a4d0-25d8c6dc44ce None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:19.726338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69df1c-3d59-4337-a4d0-25d8c6dc44ce None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:19.726338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69df1c-3d59-4337-a4d0-25d8c6dc44ce None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:19.727087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 212/423] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:19.733054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb05362-9d4c-4192-9252-7d925daac6c7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:19.733603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb05362-9d4c-4192-9252-7d925daac6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:19.734080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb05362-9d4c-4192-9252-7d925daac6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:19.754844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-eeb05362-9d4c-4192-9252-7d925daac6c7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:19.756571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eeb05362-9d4c-4192-9252-7d925daac6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:19.882177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 213/424] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:19.906643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fd4fb-6e80-4c98-a18b-c8856b3563a9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:19.944305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb05362-9d4c-4192-9252-7d925daac6c7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:19.950172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:19.951107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:19.951610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:19.952084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:19.953142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=c109cd55dcce43668065e0c553a39777) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:19.953651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:19.955269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:19.962767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 212/425] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:19 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=98ebd7033f624d02bef09757068ee356 => generated 611 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:20.098774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb05362-9d4c-4192-9252-7d925daac6c7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:20.130825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:20.153747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:20.154570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:20.154848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:20.155117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:20.174174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:20.182203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:20.183089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:20.183874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 214/426] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:20.189241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:20.205729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:20.310831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff536b9-adb2-4def-801d-da9fa750f4a8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:20.357568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6300e-d651-4307-ace4-b9a3851ede1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:20.358967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6300e-d651-4307-ace4-b9a3851ede1c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:20.359713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6300e-d651-4307-ace4-b9a3851ede1c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:20.360423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6300e-d651-4307-ace4-b9a3851ede1c None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:20.361844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6300e-d651-4307-ace4-b9a3851ede1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:20.362358 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6300e-d651-4307-ace4-b9a3851ede1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:20.364575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6300e-d651-4307-ace4-b9a3851ede1c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:20.376668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 213/427] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:45:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:20.385115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:20.492443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd3d13-4107-41f5-9e67-d5b58be5b8fa None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:20.493090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd3d13-4107-41f5-9e67-d5b58be5b8fa None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:20.493582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd3d13-4107-41f5-9e67-d5b58be5b8fa None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:20.495394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 214/428] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:20.500180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292b47b-26d4-4f2b-b8f3-ce4f3ef56eb6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:20.509443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659078-0eba-49ad-bfa1-96c529af247a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:20.509994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659078-0eba-49ad-bfa1-96c529af247a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:20.510465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659078-0eba-49ad-bfa1-96c529af247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:20.527889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-74659078-0eba-49ad-bfa1-96c529af247a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:20.529488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-74659078-0eba-49ad-bfa1-96c529af247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:20.536262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:20.537089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:20.537367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:20.537642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:20.591870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:20.598966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:20.616451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 215/429] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:20.616603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:20.616603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:20.636314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74659078-0eba-49ad-bfa1-96c529af247a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:20.721537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 215/430] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 15:45:20.729627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:20.773388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74659078-0eba-49ad-bfa1-96c529af247a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:20.815622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:20.816743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:20.817231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:20.817648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:20.828684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:20.837997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:20.838948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:20.839790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:20.840898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:20.847590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 216/431] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:45:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:20.850621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:20.861015 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ff37-0e42-458e-abe1-e8e55e76aea0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:20.903976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:20.905364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:20.905852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:20.906327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:20.907728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:20.909424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:20.912006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 216/432] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:20 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:20.920395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:20.933782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:20.992652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fcefb-d2b6-4dbd-a067-063ef6261f17 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.029428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.030473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.030964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.031461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.032450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:21.032983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:21.034662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:21.039835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98c74-0a04-414b-adc5-53a52b497f54 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.046492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 217/433] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:21.055838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.084424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.085432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.085713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.085988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.087334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:21.089434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:21.091531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 217/434] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:20 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:21.097902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.133901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:21.186500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32f7ae-a9ad-4f4f-a984-2719a6258c50 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.217619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:21.233055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.233255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962d81-3fe4-4435-bb02-8b261ddbe9a0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.234172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.234762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.235261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.269624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962857a3-ae60-4147-addb-ebf33f47134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.270342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962857a3-ae60-4147-addb-ebf33f47134b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.270751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962857a3-ae60-4147-addb-ebf33f47134b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.271029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962857a3-ae60-4147-addb-ebf33f47134b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.271851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962857a3-ae60-4147-addb-ebf33f47134b None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=98ebd7033f624d02bef09757068ee356, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:21.272169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962857a3-ae60-4147-addb-ebf33f47134b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:21.273653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962857a3-ae60-4147-addb-ebf33f47134b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:21.279809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 218/435] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=98ebd7033f624d02bef09757068ee356 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:21.281081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34df6c-ae83-48e7-bf62-176a842ba23e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.281365 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34df6c-ae83-48e7-bf62-176a842ba23e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.281644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34df6c-ae83-48e7-bf62-176a842ba23e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.283046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 219/436] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:21.288761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e02d65-9ade-42f9-b0e0-42ed23a924fe None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.289052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e02d65-9ade-42f9-b0e0-42ed23a924fe None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.289328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e02d65-9ade-42f9-b0e0-42ed23a924fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.308869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-30e02d65-9ade-42f9-b0e0-42ed23a924fe None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:21.310105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-30e02d65-9ade-42f9-b0e0-42ed23a924fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:21.322279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:21.324107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:21.389172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 220/437] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:21.413510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e02d65-9ade-42f9-b0e0-42ed23a924fe None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.432231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:21.459241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 218/438] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:21.465048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.497252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:21.534650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e02d65-9ade-42f9-b0e0-42ed23a924fe None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.574913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424cae8-4a18-42ec-b8b4-427ca40a6ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.575729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424cae8-4a18-42ec-b8b4-427ca40a6ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.576011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424cae8-4a18-42ec-b8b4-427ca40a6ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.576284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424cae8-4a18-42ec-b8b4-427ca40a6ac6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.588188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424cae8-4a18-42ec-b8b4-427ca40a6ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:21.595819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386cc91-192d-42b6-a517-01c66cb00e8d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-d424cae8-4a18-42ec-b8b4-427ca40a6ac6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:21.596587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:21.597756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:21.599519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 221/439] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:21.602663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4314d-0b56-4774-9de8-1877068f4003 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.602663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4314d-0b56-4774-9de8-1877068f4003 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.602663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4314d-0b56-4774-9de8-1877068f4003 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.604404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 222/440] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:21.604404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4314d-0b56-4774-9de8-1877068f4003 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.640870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa0983-0bd5-4f6d-a8bf-e57e543013cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.641740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa0983-0bd5-4f6d-a8bf-e57e543013cc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.642024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa0983-0bd5-4f6d-a8bf-e57e543013cc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.642287 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa0983-0bd5-4f6d-a8bf-e57e543013cc None admin] PATH_INFO: `/v3/projects/2f41ab15737f42cd803bf7f27bc66b36` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.643475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa0983-0bd5-4f6d-a8bf-e57e543013cc None admin] RBAC: Authorizing `identity:get_project(project_id=2f41ab15737f42cd803bf7f27bc66b36)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:21.644939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa0983-0bd5-4f6d-a8bf-e57e543013cc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:21.647336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 219/441] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/projects/2f41ab15737f42cd803bf7f27bc66b36 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:21.648336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778c43d-e58a-4a9b-a012-13bf55ad73e3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.648642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778c43d-e58a-4a9b-a012-13bf55ad73e3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.648912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778c43d-e58a-4a9b-a012-13bf55ad73e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.665970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a778c43d-e58a-4a9b-a012-13bf55ad73e3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:21.667143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a778c43d-e58a-4a9b-a012-13bf55ad73e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:21.700398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:21.728623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4314d-0b56-4774-9de8-1877068f4003 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.749284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 220/442] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:21.750126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778c43d-e58a-4a9b-a012-13bf55ad73e3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.769508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.770774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.771250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.771715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.772490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:21.772802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:21.773517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:21.774162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:21.782505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 223/443] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:21.783298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.892967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bca5a-160f-4f2b-852f-35df4c395744 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:21.928979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:21.929772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:21.930054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:21.930330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:21.933085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:21.941595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:21.949935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:21.950811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:21.951616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 224/444] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:21.952259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:21.979850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778c43d-e58a-4a9b-a012-13bf55ad73e3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.013158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.014032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.014311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.014584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.015730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.017176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.018778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 221/445] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:22.019528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.055765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b11d-924c-4920-a387-2f8cbb198f3d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.097881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.099659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.099855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.100118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.113008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:22.125306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30539dec-62f9-48d8-98e5-04ac7bf67fda None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.160026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.161303 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.161601 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.161873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.162610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:22.162907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.164020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:22.164269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.171484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.172814 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 222/446] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:22.172982 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.173608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.272981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683d90c-b16d-4741-8c48-d0805757e0bd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.275720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 225/447] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:22.276584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.307180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.307824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.308102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.308396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.309211 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=2f41ab15737f42cd803bf7f27bc66b36) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:22.309526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.310940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.316693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 223/448] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=2f41ab15737f42cd803bf7f27bc66b36&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:22.317462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.346598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:22.382462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:22.390287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452c023-8756-4a65-ae7e-261115a6c116 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.431303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.432074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.432382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.432655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.444659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cbc19-97cf-4606-b398-c611073a3182 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.488003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.489528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.489909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.490379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] PATH_INFO: `/v3/projects/c109cd55dcce43668065e0c553a39777` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.492644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] RBAC: Authorizing `identity:get_project(project_id=c109cd55dcce43668065e0c553a39777)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.494641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.497101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 224/449] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/projects/c109cd55dcce43668065e0c553a39777 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:22.502443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.510636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.512786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.609310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:22.621713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:22.628103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac9d76-1eb6-47cb-abf4-8419318b22eb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.630578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 226/450] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:22.636811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.672398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.673269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.673545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.673880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] PATH_INFO: `/v3/groups/847db62643e9457fbef40769ec290138` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.674979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] RBAC: Authorizing `identity:get_group(group_id=847db62643e9457fbef40769ec290138)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.677319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.680564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 225/451] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/groups/847db62643e9457fbef40769ec290138 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:22.684114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.768556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821d70-2026-415c-89d0-b2928a9101a4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.808765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7b635-2fbc-4fc5-921a-d4985e896cb6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.809780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.810649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.810936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.811210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] PATH_INFO: `/v3/projects/3e30cc2a9bc04d579b04c2060b966f87` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.812491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] RBAC: Authorizing `identity:get_project(project_id=3e30cc2a9bc04d579b04c2060b966f87)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.813942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.815653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 227/452] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/projects/3e30cc2a9bc04d579b04c2060b966f87 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:22.821901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:22.854962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.855822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.855931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.856161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.857056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] RBAC: Adding query filter params (group.id=847db62643e9457fbef40769ec290138, role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=c109cd55dcce43668065e0c553a39777) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:22.857367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.858789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.869607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 226/453] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/role_assignments?group.id=847db62643e9457fbef40769ec290138&scope.project.id=c109cd55dcce43668065e0c553a39777&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:22.895532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:22.936666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb297-26aa-4ca4-9145-60e9571124c4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:22.974097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395eabc9-4035-4a59-86dd-b7a853ce7eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:22.975237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395eabc9-4035-4a59-86dd-b7a853ce7eef None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:22.975512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395eabc9-4035-4a59-86dd-b7a853ce7eef None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:22.975782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395eabc9-4035-4a59-86dd-b7a853ce7eef None admin] PATH_INFO: `/v3/groups/44194820779141b5aa6b2cdcc4fbac78` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:22.977086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395eabc9-4035-4a59-86dd-b7a853ce7eef None admin] RBAC: Authorizing `identity:get_group(group_id=44194820779141b5aa6b2cdcc4fbac78)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:22.978558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395eabc9-4035-4a59-86dd-b7a853ce7eef None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:22.980238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 228/454] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/groups/44194820779141b5aa6b2cdcc4fbac78 => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:22.986197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:23.002899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:23.091629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0037-fea9-4179-84d9-7457227e66b0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:23.128272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5eafe-7d69-4304-b51c-21634a81bc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:23.128897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5eafe-7d69-4304-b51c-21634a81bc0e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:23.129183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5eafe-7d69-4304-b51c-21634a81bc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:23.129979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5eafe-7d69-4304-b51c-21634a81bc0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:23.130812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5eafe-7d69-4304-b51c-21634a81bc0e None admin] RBAC: Adding query filter params (group.id=44194820779141b5aa6b2cdcc4fbac78, role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=3e30cc2a9bc04d579b04c2060b966f87) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:23.131124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5eafe-7d69-4304-b51c-21634a81bc0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:23.132589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5eafe-7d69-4304-b51c-21634a81bc0e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:23.138848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 227/455] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Jun 7 15:45:22 2022] GET /identity/v3/role_assignments?group.id=44194820779141b5aa6b2cdcc4fbac78&scope.project.id=3e30cc2a9bc04d579b04c2060b966f87&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:26.120795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18fb7-768b-4c93-abc3-2927557c24c0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.121854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18fb7-768b-4c93-abc3-2927557c24c0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.122428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18fb7-768b-4c93-abc3-2927557c24c0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.124346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 229/456] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:26.129715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd963-4d6f-4d16-99eb-11fcb7c89180 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.130648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd963-4d6f-4d16-99eb-11fcb7c89180 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.131126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd963-4d6f-4d16-99eb-11fcb7c89180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.137663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebee966-d312-451e-8f79-b664f922ec4b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.138216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebee966-d312-451e-8f79-b664f922ec4b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.138689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebee966-d312-451e-8f79-b664f922ec4b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.140553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 230/457] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:45:26.145696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf262a-e75b-448d-a776-e894ff7164c1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.146937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf262a-e75b-448d-a776-e894ff7164c1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.147227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf262a-e75b-448d-a776-e894ff7164c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.165364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-60efd963-4d6f-4d16-99eb-11fcb7c89180 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:26.165938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d5cf262a-e75b-448d-a776-e894ff7164c1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:26.166554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-60efd963-4d6f-4d16-99eb-11fcb7c89180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:26.167521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d5cf262a-e75b-448d-a776-e894ff7164c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:26.270539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 231/458] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:26.271916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a68bc-4120-45f6-a23d-b85c929758f8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.272205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a68bc-4120-45f6-a23d-b85c929758f8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.272535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a68bc-4120-45f6-a23d-b85c929758f8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.274127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 232/459] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:26.279285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4db0a-92aa-4857-a17b-9add2d8a061a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.279569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4db0a-92aa-4857-a17b-9add2d8a061a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.279850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4db0a-92aa-4857-a17b-9add2d8a061a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.299788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b3d4db0a-92aa-4857-a17b-9add2d8a061a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:26.301032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b3d4db0a-92aa-4857-a17b-9add2d8a061a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:26.310734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 228/460] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:26.318924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efd963-4d6f-4d16-99eb-11fcb7c89180 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:26.339545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:26.393388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:26.412445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 233/461] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:26.413315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4db0a-92aa-4857-a17b-9add2d8a061a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:26.457027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efd963-4d6f-4d16-99eb-11fcb7c89180 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:26.495756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:26.496794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.497081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.497357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.508587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:26.510109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:26.531222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:26.532104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:26.532966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:26.532966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:26.532966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:26.532966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:26.537919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 229/462] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:26.539501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:26.548050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4db0a-92aa-4857-a17b-9add2d8a061a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:26.592891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:26.593977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.594263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.594538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.603374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:26.606794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:26.608262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:26.627140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:26.628039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:26.628889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:26.628889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:26.628889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:26.628889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:26.628889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 234/463] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:26.629161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:26.653090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc997ea5-6706-48cf-922f-8100e449c30c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:26.678324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:26.692904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6f54e-ba5c-433d-a53b-d83cbea78a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:26.693822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6f54e-ba5c-433d-a53b-d83cbea78a4f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.694180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6f54e-ba5c-433d-a53b-d83cbea78a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.694458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6f54e-ba5c-433d-a53b-d83cbea78a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.706228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6f54e-ba5c-433d-a53b-d83cbea78a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:26.707111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6f54e-ba5c-433d-a53b-d83cbea78a4f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-17e6f54e-ba5c-433d-a53b-d83cbea78a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:26.726205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:26.727090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:26.727947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:26.727947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:26.727947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:26.727947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:26.727947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 230/464] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:26.728701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca24d-383c-4b58-bc36-58c52734ec0c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.728957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca24d-383c-4b58-bc36-58c52734ec0c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.729269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca24d-383c-4b58-bc36-58c52734ec0c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.730548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 231/465] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:26.732221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca24d-383c-4b58-bc36-58c52734ec0c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:26.743898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf9c4-c9ce-4f0b-836c-f4070abfc62f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:26.780165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:26.781138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.781591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.782035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.782943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:26.783429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:26.784893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:26.797627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 235/466] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:26.802339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:26.850960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca24d-383c-4b58-bc36-58c52734ec0c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:26.866902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:26.890940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95da0a-095c-4f63-b5ad-2e8c01c92a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:26.891681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95da0a-095c-4f63-b5ad-2e8c01c92a0f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.892026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95da0a-095c-4f63-b5ad-2e8c01c92a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.892678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95da0a-095c-4f63-b5ad-2e8c01c92a0f None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.893194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95da0a-095c-4f63-b5ad-2e8c01c92a0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:26.893493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95da0a-095c-4f63-b5ad-2e8c01c92a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:26.894869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95da0a-095c-4f63-b5ad-2e8c01c92a0f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:26.911418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 232/467] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:26.911418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2321ae2-73d9-4619-aed8-12c587f0371e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.911418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2321ae2-73d9-4619-aed8-12c587f0371e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.911418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2321ae2-73d9-4619-aed8-12c587f0371e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.918460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045866d8-3262-40ce-914a-deeb1396ce95 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:26.921754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:26.924949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c2321ae2-73d9-4619-aed8-12c587f0371e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:26.926160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c2321ae2-73d9-4619-aed8-12c587f0371e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:26.955870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:26.956570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:26.956858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:26.957141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:26.957928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:26.958241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:26.959796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:26.977376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 236/468] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:26.977376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.001485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 233/469] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:27.009510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2321ae2-73d9-4619-aed8-12c587f0371e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.085064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c2723-f41f-4690-a829-4614020b8fd7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.117892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2321ae2-73d9-4619-aed8-12c587f0371e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.119584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:27.121254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.121936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.122219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.122661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.123204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:27.124937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:27.146653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 237/470] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:45:26 2022] POST /identity/v3/users => generated 248 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:27.147800 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.180135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.181342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:27.181774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.182059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.182334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.183069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:27.184577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:27.205474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 234/471] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:45:27 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:27.206082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.283446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bdfbc-40f3-424b-95e2-75c06a3c1e15 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.335190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f09d92-17c2-48dc-94ba-53ea3f26356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.335943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f09d92-17c2-48dc-94ba-53ea3f26356f None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.336314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f09d92-17c2-48dc-94ba-53ea3f26356f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.336626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f09d92-17c2-48dc-94ba-53ea3f26356f None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.338154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f09d92-17c2-48dc-94ba-53ea3f26356f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:27.340023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f09d92-17c2-48dc-94ba-53ea3f26356f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:27.352736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bce78b-0dfc-4a4d-aba6-6234b0d79506 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.377360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 238/472] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:45:27 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:27.379805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f78487-7bc1-4348-b256-4bf080daab59 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.380093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f78487-7bc1-4348-b256-4bf080daab59 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.380420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f78487-7bc1-4348-b256-4bf080daab59 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.382557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 239/473] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:27.389190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0b590-2b18-4ef4-b16e-d7982fa5e632 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.389561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0b590-2b18-4ef4-b16e-d7982fa5e632 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.389893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0b590-2b18-4ef4-b16e-d7982fa5e632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.402209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.403268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.403747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.404164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.413679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-dba0b590-2b18-4ef4-b16e-d7982fa5e632 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:27.414993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dba0b590-2b18-4ef4-b16e-d7982fa5e632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:27.420811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:27.422137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:27.443698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:27.444819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:27.445833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:27.445833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:27.445833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:27.445833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:27.445833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 235/474] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:27.451030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.472313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:27.529624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 240/475] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:27.556895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0b590-2b18-4ef4-b16e-d7982fa5e632 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.565547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:27.593861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bb575-35eb-4764-9b7a-f2b2ef45ec18 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.632512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.633265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.633464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.633738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.634470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:27.634785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:27.636134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:27.650510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 236/476] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:27.655848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.681284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0b590-2b18-4ef4-b16e-d7982fa5e632 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.728585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.729368 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.729659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.729937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.731030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:27.732726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:27.741003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.sql.core [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 15:45:27.741003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 15:45:27.741003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [parameters: {'id': '1adc0009bd2447c09c26101c786d0a7f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 15:45:27.741003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80320) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 15:45:27.743581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jun 07 15:45:27.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 15:45:27.767760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application yield resource Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise self.value Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application session.commit() Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 15:45:27.773108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.flush() Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application compat.raise_( Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 15:45:27.773937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 15:45:27.774794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 15:45:27.775608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [parameters: {'id': '1adc0009bd2447c09c26101c786d0a7f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:27.777213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 15:45:27.779081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:27.781214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 241/477] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Jun 7 15:45:27 2022] POST /identity/v3/roles => generated 144 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:27.786136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:27.797603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dba739-770a-4e92-a381-bb754faf33be None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.846622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.847543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.847829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.848690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.850131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:27.851569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:27.879568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 237/478] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:45:27 2022] POST /identity/v3/users => generated 249 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:45:27.892757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:27.927947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c9ce0-b62d-46dc-9517-2c3e702fc37c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:27.967785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882a7e1-516a-4b52-85ae-8d3314fe1248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:27.968868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882a7e1-516a-4b52-85ae-8d3314fe1248 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:27.969500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882a7e1-516a-4b52-85ae-8d3314fe1248 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:27.969915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882a7e1-516a-4b52-85ae-8d3314fe1248 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:27.982894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882a7e1-516a-4b52-85ae-8d3314fe1248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f882a7e1-516a-4b52-85ae-8d3314fe1248 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:27.992803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:27.993956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 15:45:27.994953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:27.997233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 242/479] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:45:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:28.003282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:28.149892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6eb8-e30a-4d6d-84ef-d7860b622b19 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:28.198668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:28.209714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16420203-71c1-4f2c-9393-ea4115be3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:28.210588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16420203-71c1-4f2c-9393-ea4115be3548 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:28.211098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16420203-71c1-4f2c-9393-ea4115be3548 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:28.211606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16420203-71c1-4f2c-9393-ea4115be3548 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:28.212388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16420203-71c1-4f2c-9393-ea4115be3548 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:28.212720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16420203-71c1-4f2c-9393-ea4115be3548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:28.214102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16420203-71c1-4f2c-9393-ea4115be3548 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:28.224144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 238/480] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:29.850978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02322a1-829d-43d6-b321-237c450a6204 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:29.852284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02322a1-829d-43d6-b321-237c450a6204 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:29.852893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02322a1-829d-43d6-b321-237c450a6204 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:29.854904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 243/481] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:29.861252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a418ccc-95cb-4f2a-acc0-b6310a590663 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:29.862221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a418ccc-95cb-4f2a-acc0-b6310a590663 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:29.862742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a418ccc-95cb-4f2a-acc0-b6310a590663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:29.884048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0a418ccc-95cb-4f2a-acc0-b6310a590663 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:29.885614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0a418ccc-95cb-4f2a-acc0-b6310a590663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:29.989295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 239/482] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:30.016773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02322a1-829d-43d6-b321-237c450a6204 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:30.126949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02322a1-829d-43d6-b321-237c450a6204 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:30.160807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:30.169108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:30.169909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.170225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.170567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.185522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:30.193674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:30.194575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:30.195349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:30.200856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 244/483] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:30.206361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a418ccc-95cb-4f2a-acc0-b6310a590663 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:30.325198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a418ccc-95cb-4f2a-acc0-b6310a590663 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:30.347405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:30.369908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec3839-3594-481c-ad99-c8511e95bb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:30.370575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec3839-3594-481c-ad99-c8511e95bb8f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.370856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec3839-3594-481c-ad99-c8511e95bb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.371131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec3839-3594-481c-ad99-c8511e95bb8f None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.371936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec3839-3594-481c-ad99-c8511e95bb8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:30.372239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec3839-3594-481c-ad99-c8511e95bb8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:30.373625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec3839-3594-481c-ad99-c8511e95bb8f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:30.387698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 240/484] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:30.393260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:30.435814 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc44f6b-1144-44a1-8036-0c422eca4165 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.436115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc44f6b-1144-44a1-8036-0c422eca4165 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.436377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc44f6b-1144-44a1-8036-0c422eca4165 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.438119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 241/485] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:30.443135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8cbf-bf45-4d06-bd9d-aa4319225e90 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.443411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8cbf-bf45-4d06-bd9d-aa4319225e90 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.443689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8cbf-bf45-4d06-bd9d-aa4319225e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.471243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8b7b8cbf-bf45-4d06-bd9d-aa4319225e90 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:30.474502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8b7b8cbf-bf45-4d06-bd9d-aa4319225e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:30.544269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085754c-188a-42fc-a2dc-6f543c279024 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:30.581809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 242/486] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:30.583825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed472ea-3d20-4bed-89cf-0bd40359ee99 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.584113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed472ea-3d20-4bed-89cf-0bd40359ee99 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.584426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed472ea-3d20-4bed-89cf-0bd40359ee99 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.585324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:30.585954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 243/487] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:30.586179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.586465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.586739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.591911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21d1b3-5f9e-4be6-9468-a6039eef89ec None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.592196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21d1b3-5f9e-4be6-9468-a6039eef89ec None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.592511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21d1b3-5f9e-4be6-9468-a6039eef89ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.609599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:30.611100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:30.619172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-df21d1b3-5f9e-4be6-9468-a6039eef89ec None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:30.620755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-df21d1b3-5f9e-4be6-9468-a6039eef89ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:30.631948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:30.632801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:30.633631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:30.633631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:30.633631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:30.633631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:30.635802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 245/488] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:30.636811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:30.686054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:30.712439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 244/489] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:30.713537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21d1b3-5f9e-4be6-9468-a6039eef89ec None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:30.759866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486c2f3-86ba-4add-b611-e11997d89cf3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:30.788015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:30.807197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:30.808113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.808483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.808833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.821468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:30.829704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:30.833537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 246/490] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:30.834544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:30.873102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21d1b3-5f9e-4be6-9468-a6039eef89ec None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:30.909742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:30.910637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:30.911063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:30.911433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:30.912307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:30.912754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:30.914219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:30.931077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 245/491] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:30.932128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:30.970589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1d443-5ad8-4ed0-86f2-c77869a077d1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:30.972166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.009459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.010257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.010538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.010811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.023305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:31.031110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:31.031961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:31.032774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 247/492] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:31.033947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.041926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.045222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393dbaf-7c0a-4044-9731-1a33c532a000 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.085839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1264a1f-290e-4be3-80c4-10fefe20e521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.086664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1264a1f-290e-4be3-80c4-10fefe20e521 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.086943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1264a1f-290e-4be3-80c4-10fefe20e521 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.087215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1264a1f-290e-4be3-80c4-10fefe20e521 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.087956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1264a1f-290e-4be3-80c4-10fefe20e521 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:31.088258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1264a1f-290e-4be3-80c4-10fefe20e521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.089700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1264a1f-290e-4be3-80c4-10fefe20e521 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.098029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 246/493] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:31.099909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891245-4c74-4158-9b23-452599da85ea None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.100197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891245-4c74-4158-9b23-452599da85ea None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.100505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891245-4c74-4158-9b23-452599da85ea None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.104608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 247/494] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:31.104608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891245-4c74-4158-9b23-452599da85ea None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.151086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa50ef-b277-4339-aedb-163ae1600934 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.194373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.195164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.195439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.195708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.203213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.204667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.216884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891245-4c74-4158-9b23-452599da85ea None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:31.220479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:31.221442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:31.222337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 248/495] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:31.222979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.247705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.251081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44252d8-22f0-46a9-9f20-6e250d7cc6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.251750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44252d8-22f0-46a9-9f20-6e250d7cc6dd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.252028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44252d8-22f0-46a9-9f20-6e250d7cc6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.252303 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44252d8-22f0-46a9-9f20-6e250d7cc6dd None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.253076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44252d8-22f0-46a9-9f20-6e250d7cc6dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:31.253377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44252d8-22f0-46a9-9f20-6e250d7cc6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.255350 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44252d8-22f0-46a9-9f20-6e250d7cc6dd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.263173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 248/496] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:31.264502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e6057-2c9c-4645-9101-d303e5dab20c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.264789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e6057-2c9c-4645-9101-d303e5dab20c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.265066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e6057-2c9c-4645-9101-d303e5dab20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.281576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c25e6057-2c9c-4645-9101-d303e5dab20c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:31.282751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c25e6057-2c9c-4645-9101-d303e5dab20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:31.299608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.322981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c3fb9-99cc-46a6-b492-32eb7806d9b5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.349442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 249/497] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:31.350906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6c15-941d-4908-bfb8-b584c15e93ee None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.351192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6c15-941d-4908-bfb8-b584c15e93ee None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.351463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6c15-941d-4908-bfb8-b584c15e93ee None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.352748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 250/498] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:31.353431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f6c15-941d-4908-bfb8-b584c15e93ee None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.364796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.365641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.365930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.366204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.377640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.379623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:31.397006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:31.398057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:31.398987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:31.398987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:31.398987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:31.398987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:31.398987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 249/499] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:31.398987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.458699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f6c15-941d-4908-bfb8-b584c15e93ee None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.477705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.494049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a57370-c764-476c-92b8-4c13554be077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.494708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a57370-c764-476c-92b8-4c13554be077 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.494981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a57370-c764-476c-92b8-4c13554be077 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.495252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a57370-c764-476c-92b8-4c13554be077 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.496076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a57370-c764-476c-92b8-4c13554be077 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:31.496402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a57370-c764-476c-92b8-4c13554be077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.497815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a57370-c764-476c-92b8-4c13554be077 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.502643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091526c9-7bc7-4608-89f6-abd3c9374360 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.534195 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 251/500] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:31.534993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.535972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90606c-1af5-4c39-8b99-78cd55660f7e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.536277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.536470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90606c-1af5-4c39-8b99-78cd55660f7e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.536590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.536858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.536961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90606c-1af5-4c39-8b99-78cd55660f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.547833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.549319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.554459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5e90606c-1af5-4c39-8b99-78cd55660f7e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:31.555626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e90606c-1af5-4c39-8b99-78cd55660f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:31.563424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:31.570025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:31.570857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:31.571675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:31.571675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:31.571675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:31.571675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:31.571675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 250/501] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:31.572098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.627732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 252/502] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:31.628398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90606c-1af5-4c39-8b99-78cd55660f7e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.673413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1887-6296-4bf0-b268-258b8efe3036 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.697872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.707028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.708542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.708831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.709116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.718978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:31.726709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90606c-1af5-4c39-8b99-78cd55660f7e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:31.727793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:31.728686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:31.729498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 251/503] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:31.730135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.762992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.763889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.764170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.764471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.765181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:31.765478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.766830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.778912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 253/504] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:31.779677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.782230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.827544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881479b-310c-4cf6-9663-31e44fede3ed None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.861637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.862280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.862561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.862833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.885155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd9c4c-a60c-41e7-94f7-39a187ef7762 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:31.917856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:31.919400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:31.920088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:31.920376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:31.920679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:31.921396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:31.921699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.923050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.931689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:31.933237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:31.935215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 254/505] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:31.936008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:31.995640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.031107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 252/506] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:32.031904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.033654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff520a7-fef0-4478-a7bb-5876da084ecf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.069209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.070055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.070331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.070600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.081573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.083036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:32.101004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:32.101833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:32.102666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:32.102666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:32.102666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:32.102666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:32.102666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 255/507] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:32.103454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.122416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.140761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382fca0-aadc-4a1b-9721-d49cf8a40cb9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.177769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.178443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.178783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.179071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.179826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:32.180135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.181562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.189506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 253/508] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:32.190496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.206380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ffb45-ee33-46fc-accc-42cbb76ac74b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.220404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.240457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.241247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.241532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.241810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.249208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.250589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:32.266064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:32.267120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:32.268061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 256/509] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:32.268770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.292135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66e931-0742-41a5-a95e-c5bb78bd1de5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.324758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.325548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.325835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.326103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.333453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.334887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:32.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:32.350472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:32.351644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 254/510] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:32.352484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.362165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.374503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf813e-d9c8-458a-b19a-5ff45f574395 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.409446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.410229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.411826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.412105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.422355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.423783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:32.440951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:32.441902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:32.442728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:32.442728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:32.442728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:32.442728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:32.442934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 257/511] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:32.443517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.453798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3db308-9d97-4ad7-a31d-b7967a32d4d5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.471068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.492312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.493005 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.493288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.493564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.494366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:32.494670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.496083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.510169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 255/512] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:32.510913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.551419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43a98-af76-4bfa-9ee0-50e5d91fe902 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.578278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.584678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.585465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.585746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.586018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.596317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.597800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.612307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e3e4a-43f6-4bdb-be94-43b1c00d0a82 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:32.615476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:32.616509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:32.617386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:32.617386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:32.617386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:32.617386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:32.617386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 258/513] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:32.617867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.646170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.646840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.647115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.647387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.648209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:32.648546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.649965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.686634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 256/514] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:32.689284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.696249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.718124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725d8f3-368c-49f8-9cfb-553f80e46be1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.751010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.751660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.751935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.752205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.753056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:32.753359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.755943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.774000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 259/515] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:32.774622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.795216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f57ac-73a8-4e20-a843-2660642f8e07 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.820977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.830192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.830848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.831129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.831403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.832118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:32.832478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.833864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.844978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 257/516] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:32.845765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.879598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c81612-fc9e-44cb-945f-e0903c2bd7a5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.916222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.916992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.917275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.917546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.918233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.919636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:32.927549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:32.939458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 260/517] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:45:32 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:32.940245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:32.948626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e99360-dc6b-4828-a254-c33d20a7cb70 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:32.990445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:32.991117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:32.991400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:32.991675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:32.992419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:32.992725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:32.994123 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.070065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225dcc-b583-4769-9f3f-5c6d8e865995 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.109172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:33.109844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:33.110122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:33.110391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:33.145393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 258/518] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:33.145393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.193322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.209205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:33.210672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.213586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.268236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963af0e-7600-4d51-8653-c8fd11563ed1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.306542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:33.307468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:33.307748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:33.308180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:33.322388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 261/519] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:33.322388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.386220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:33.387672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.414924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.426522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61da125-22fd-4cc4-96e1-f69e97c521b4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.437139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.465647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:33.466444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:33.466721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:33.466991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:33.475018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:33.476486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:33.496558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:33.497581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:33.498607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:33.499119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 259/520] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:33.499512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.500978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 262/521] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:33.501957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.616091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615b699-8895-4bd5-aba3-81d1c0371a2d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.617038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ad361-dcbc-43df-be64-88fe45e60991 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.651123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:33.651912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:33.652197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:33.652502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:33.655629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:33.656639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:33.657071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:33.657490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:33.662883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:33.664332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.665752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:33.667499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:33.681739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:33.682781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:33.688470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:33.688470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:33.688470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:33.688470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:33.688470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 263/522] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:33.688784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:33.689977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 15:45:33.690862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:33.691356 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.691422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 260/523] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:33.691422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.693095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.700934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.795741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566845e-dfba-4bf6-b694-dcbf37f9f43d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.798736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f30dc-75b9-4170-959a-0f952ba18630 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.829051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:33.829718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:33.830023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:33.830313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:33.831083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:33.831395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:33.833147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.838243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:33.839226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:33.839644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:33.840065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:33.852564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:33.852715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 261/524] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:33.854348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:33.858262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:33.872349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:33.873249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:33.874074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:33.874074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:33.874074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:33.874074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:33.874074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 264/525] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:33.874348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:33.931297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.934288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:33.971179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dde66-e6eb-4783-ab78-ba969f919c20 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:33.980619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f34b-3e5a-44bb-8fa0-77daaa5185ba None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.007499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.008156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.008469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.008750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.009459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:34.009771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.011128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.015344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.016009 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.016292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.016599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.017436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:34.017745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.019155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.023533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 262/526] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:34.024545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.040402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 265/527] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Jun 7 15:45:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:34.040402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.143767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef531102-b6e4-4375-8b12-bd4efd98a184 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.144343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3657f8a-c921-43f8-9c4e-878ec73d05e4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.163836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:34.175553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:34.185551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.186228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.186511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.186766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.187620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=ac0206e09e4f426d9f4b54543d89c039) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:34.187948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.189327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.189443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.189988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.190271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.190544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.191262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:34.191629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.193019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.195846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 263/528] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/role_assignments?user.id=ac0206e09e4f426d9f4b54543d89c039&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:34.201397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.210138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 266/529] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:34.211420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.321653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb4d17-b30e-4d61-bfe2-fd53366ceb2b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.363118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054f55c-ef8e-4173-a112-63d43efe062f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.365131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.367548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.368070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.368608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.378835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.382266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.401990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.402559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.402842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.403118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:34.403808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:34.404622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:34.405489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 264/530] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:34.406353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.461854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:34.470519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:34.501812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.503723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.528894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e4f50-3025-4d83-b2f3-00fadc351c7b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.565972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.567451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.567855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.568225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.575774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.577560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:34.597241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:34.599040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 265/531] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:34.601274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.645741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 267/532] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:34.646610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.718435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1497a52-a1e5-4842-8912-f7446a705293 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.735908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:34.746029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:34.773348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.774019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.774128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c682f8-3177-4de8-895e-98a77082758c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.774303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.774581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.775373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:34.775683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.778409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.794711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 266/533] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:34.795638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.819677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.820350 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.820662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.820941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.821734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:34.822065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.824347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.850024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 268/534] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:34.851516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:34.909509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a12b8-6f8c-4de8-b3de-01ef5d3d9699 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.949642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:34.950661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:34.951040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:34.951412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:34.963879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:34.965556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:34.989125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28eacd-5f20-4ac7-bec1-debb12189e99 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:34.993724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:34.994745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:34.996335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:34.996335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:34.996335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:34.996335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:35.000542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 267/535] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:35.001483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.032512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:35.033186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.033759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.034044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.034803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:35.034980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0b352e6a51fc4d1f8b9b97d092d11bd7) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:35.035177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:35.038470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:35.040909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:35.047856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 269/536] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:34 2022] GET /identity/v3/role_assignments?user.id=0b352e6a51fc4d1f8b9b97d092d11bd7&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:35.048783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.187227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632941f0-b7d7-4381-9696-573c6cb60c26 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:35.251783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237eec2-f642-4220-8b2e-c92647fae5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:35.251783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237eec2-f642-4220-8b2e-c92647fae5ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.251783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237eec2-f642-4220-8b2e-c92647fae5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.251783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237eec2-f642-4220-8b2e-c92647fae5ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.252527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237eec2-f642-4220-8b2e-c92647fae5ec None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=a31ce1720fa84c43aec2ea016371f475) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:35.252669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237eec2-f642-4220-8b2e-c92647fae5ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:35.255722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1df07-b2d4-4a56-a51d-f33a60a13c94 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:35.255927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237eec2-f642-4220-8b2e-c92647fae5ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:35.267495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 268/537] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/role_assignments?user.id=a31ce1720fa84c43aec2ea016371f475&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:35.271862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9680da-9950-4d34-9dc2-bfa0281a5ded None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.272223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9680da-9950-4d34-9dc2-bfa0281a5ded None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.272473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9680da-9950-4d34-9dc2-bfa0281a5ded None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.273876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 269/538] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:35.279891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed64856-d0b1-40b4-b863-2824dc4c485b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.280171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed64856-d0b1-40b4-b863-2824dc4c485b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.280394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed64856-d0b1-40b4-b863-2824dc4c485b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.295112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:35.295807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.296091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.296383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.297175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:35.297484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:35.298903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:35.322882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2ed64856-d0b1-40b4-b863-2824dc4c485b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:35.324047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2ed64856-d0b1-40b4-b863-2824dc4c485b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:35.331133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 270/539] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:35.337589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.415100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:35.468385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343f6e4-692a-4c76-8a1a-ba7da5082a5c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:35.511876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:35.512615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.512899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.513175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.526359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:35.527882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:35.539343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 270/540] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:35.548043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:35.548934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:35.549766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 271/541] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:35.555615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed64856-d0b1-40b4-b863-2824dc4c485b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.577754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.590249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:35.694597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a35c26-881d-4ada-baa6-05ff45d3c897 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:35.728786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:35.737035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:35.737833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.738113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.738418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.758291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:35.766176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:35.767029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:35.767809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:35.768929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 272/542] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:35.773427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.776696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed64856-d0b1-40b4-b863-2824dc4c485b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:35.840382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:35.841062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.841343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.841611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.842407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:35.842711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:35.844484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:35.867826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 271/543] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:35.885930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abfe67-b4d3-4f47-99e5-3c1d50a9d484 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:35.889398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.921425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:35.922097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:35.922378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:35.922656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:35.923557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:35.923804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:35.925253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:35.934860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 273/544] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:35.940280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:35.983039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:36.000788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:36.007912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa93cc-42f9-4079-be95-f476f8f63a91 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:36.052457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92460b68-3985-4323-9363-0e8ba26bb401 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:36.062083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:36.062760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:36.063152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:36.063429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:36.064270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=dba9f1fa257a4f44ae159850f9ba349c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:36.064613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:36.066095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:36.078782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 272/545] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/role_assignments?user.id=dba9f1fa257a4f44ae159850f9ba349c&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:36.106086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:36.107285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:36.107762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:36.108229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:36.124554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:36.126419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:36.146418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:36.147317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:36.148588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:36.148588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:36.148588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:36.148588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:36.151223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 274/546] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:36.162064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:36.342309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc35ea-6982-4182-ad45-af8208faafd8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:36.388976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:36.389647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:36.389933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:36.390212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:36.390925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:36.391232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:36.392636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:36.404711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:36.409023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 273/547] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:36.415404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:36.577523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:36.587632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba352acf-5313-4c16-97e8-22cb66684f6c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:36.622053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:36.622767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:36.623045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:36.623315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:36.631602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:36.633072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:36.648451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:36.649885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:36.650931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 275/548] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:36.655715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:36.807113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8446c-54a7-4448-a182-0169671517a5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:36.862417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00256a95-91cc-468f-aaf2-e133ca04caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:36.863100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00256a95-91cc-468f-aaf2-e133ca04caed None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:36.863449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00256a95-91cc-468f-aaf2-e133ca04caed None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:36.863728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00256a95-91cc-468f-aaf2-e133ca04caed None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:36.864641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00256a95-91cc-468f-aaf2-e133ca04caed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:36.864953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00256a95-91cc-468f-aaf2-e133ca04caed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:36.866439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00256a95-91cc-468f-aaf2-e133ca04caed None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:36.892653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 274/549] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Jun 7 15:45:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:36.898678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:37.037837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359995f-bc20-4340-a3c7-1911b34918f0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:37.074403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ff543-8bab-4317-9be3-118ab038124b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:37.074770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:37.075119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ff543-8bab-4317-9be3-118ab038124b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:37.075424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:37.075707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:37.075849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ff543-8bab-4317-9be3-118ab038124b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:37.075978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:37.077754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 275/550] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:37.083235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be29a64-8729-40a2-b0b5-19858c9db81d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:37.083781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be29a64-8729-40a2-b0b5-19858c9db81d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:37.084254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be29a64-8729-40a2-b0b5-19858c9db81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:37.110779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3be29a64-8729-40a2-b0b5-19858c9db81d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:37.112494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3be29a64-8729-40a2-b0b5-19858c9db81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:37.129042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:37.212558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:37.212756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:37.214124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:37.241821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 276/551] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:37.265771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be29a64-8729-40a2-b0b5-19858c9db81d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:37.373858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be29a64-8729-40a2-b0b5-19858c9db81d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:37.429327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 276/552] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:37.433405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:37.444510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:37.445661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:37.446648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:37.447156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:37.481369 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:37.496387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:37.497514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:37.498328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:37.498967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:37.501457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 277/553] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:37.508176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:37.579844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8128b-76ca-4df7-88ca-b2b84b5c296f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:37.625312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:37.626256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:37.626457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:37.626555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:37.627100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:37.639917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:37.641421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:37.662301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:37.663223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:37.664804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:37.664804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:37.664804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:37.664804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:37.664804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 277/554] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:37.673247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:37.715785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0ba08-94ec-4346-b5f5-2ee3dcf8bbd9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:37.752179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:37.752937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:37.753225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:37.753503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:37.754788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:37.755191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:37.756591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:37.765511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 278/555] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:37.773480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:37.919340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8fcab-82ab-42da-b8c4-ac655bbdd1c3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:37.936840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:37.976006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2474bd8-e983-4bf8-a3d0-febed4c70f62 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:37.998076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:38.000571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badd3b0-671b-4dca-b83a-f8dc786aadc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.002396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd3b0-671b-4dca-b83a-f8dc786aadc2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.004701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd3b0-671b-4dca-b83a-f8dc786aadc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.005230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd3b0-671b-4dca-b83a-f8dc786aadc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.006299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badd3b0-671b-4dca-b83a-f8dc786aadc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:38.007208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badd3b0-671b-4dca-b83a-f8dc786aadc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.008791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badd3b0-671b-4dca-b83a-f8dc786aadc2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.032967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 278/556] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:38.034446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c43b94-7249-4736-a65f-394e6804ebf8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.034733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c43b94-7249-4736-a65f-394e6804ebf8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.035015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c43b94-7249-4736-a65f-394e6804ebf8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.041453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 279/557] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:38.044480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c19dd-e73b-4b1f-82e4-a9c5ca57117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.045832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c19dd-e73b-4b1f-82e4-a9c5ca57117f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.046340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c19dd-e73b-4b1f-82e4-a9c5ca57117f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.046555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525a02b-f106-4d95-a46c-ced4ad65e4f4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.046791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525a02b-f106-4d95-a46c-ced4ad65e4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.047085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525a02b-f106-4d95-a46c-ced4ad65e4f4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.047521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c19dd-e73b-4b1f-82e4-a9c5ca57117f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.048555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 280/558] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:38.053771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525a02b-f106-4d95-a46c-ced4ad65e4f4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.062291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c19dd-e73b-4b1f-82e4-a9c5ca57117f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.064670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c19dd-e73b-4b1f-82e4-a9c5ca57117f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-c19c19dd-e73b-4b1f-82e4-a9c5ca57117f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:38.087512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:38.089032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:38.090081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:38.090081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:38.090081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:38.090081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:38.095993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 279/559] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:38.095993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3bd28-558f-4477-893c-f409e5a7a3ac None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.095993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3bd28-558f-4477-893c-f409e5a7a3ac None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.095993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3bd28-558f-4477-893c-f409e5a7a3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.113084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-05b3bd28-558f-4477-893c-f409e5a7a3ac None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:38.114890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-05b3bd28-558f-4477-893c-f409e5a7a3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:38.197852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 280/560] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:38.204568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d264bc9-6701-4046-be7e-e2cea62b890f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.204568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d264bc9-6701-4046-be7e-e2cea62b890f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.204568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d264bc9-6701-4046-be7e-e2cea62b890f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.220127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1d264bc9-6701-4046-be7e-e2cea62b890f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:38.221631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1d264bc9-6701-4046-be7e-e2cea62b890f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:38.232976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525a02b-f106-4d95-a46c-ced4ad65e4f4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.274781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.275944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.276289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.276796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.285675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.287556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:38.303921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:38.305053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 281/561] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:38.305960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.310282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 281/562] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:38.311128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d264bc9-6701-4046-be7e-e2cea62b890f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.338623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:38.342885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:38.419667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b41ab2-6c8a-4c51-96d1-b5f4fbcecfa2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.431869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d264bc9-6701-4046-be7e-e2cea62b890f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.456493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.457180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.457491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.457774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.458446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:38.458739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.460114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.470463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.471261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.471541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.471816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.476976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 282/563] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:38.477670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.482951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:38.491480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:38.492465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:38.493333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 282/564] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:38.494023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.574337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:38.582864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:38.591382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e35955-9215-4263-b37c-242394b49b7e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.603129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e172558-67a2-49a3-81ef-2a593fa24e2c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.626228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.626890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.627172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.627448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.628229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:38.628566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.630824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.640180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.641070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.641351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.641625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.648994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 283/565] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:38.648994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.657969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:38.666474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:38.667443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:38.668257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 283/566] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:38.668961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.754705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45541768-269c-4d27-813a-cf15ed5e6347 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.775203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1192108-2527-477b-90ec-886635d9a8b9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.788413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.789210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.789494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.789771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.797103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.798540 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.809579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.810238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.810516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.810786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.811515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:38.811817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.813192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:38.813726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:38.814721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 284/567] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:38.815686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.821202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:38.825869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 284/568] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:38.826680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.830130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:38.919510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995812e-5cf3-43e1-841c-01730ac16e3f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.926560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2ef19-081e-4ee6-ac56-cfab7f2485e7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:38.952935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47492-29ac-4705-9bae-6cdad17527d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.953579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47492-29ac-4705-9bae-6cdad17527d4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.953862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47492-29ac-4705-9bae-6cdad17527d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.954136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47492-29ac-4705-9bae-6cdad17527d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.954957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c47492-29ac-4705-9bae-6cdad17527d4 None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=31137a8197ab4072b47646b893a9733a) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:38.955265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c47492-29ac-4705-9bae-6cdad17527d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.956705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c47492-29ac-4705-9bae-6cdad17527d4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.960704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:38.961360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.961638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.961910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.962636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:38.962760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 285/569] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/role_assignments?user.id=31137a8197ab4072b47646b893a9733a&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:38.962930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:38.963832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86096fe6-2c34-4efe-89c9-6ad72ac715b6 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:38.964115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86096fe6-2c34-4efe-89c9-6ad72ac715b6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:38.964270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:38.964450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86096fe6-2c34-4efe-89c9-6ad72ac715b6 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:38.965655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 286/570] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:38.970699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86096fe6-2c34-4efe-89c9-6ad72ac715b6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:38.979050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 285/571] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:38.979050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.049330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.055177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.090142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714ee30-2042-4086-9968-87bc46b7b306 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.097277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86096fe6-2c34-4efe-89c9-6ad72ac715b6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.139408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.140223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.140616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.140819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.143882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107d738-0ec3-45e7-83ca-ca2d3e9364d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.148000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d738-0ec3-45e7-83ca-ca2d3e9364d5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.148000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d738-0ec3-45e7-83ca-ca2d3e9364d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.148000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d738-0ec3-45e7-83ca-ca2d3e9364d5 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.148000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107d738-0ec3-45e7-83ca-ca2d3e9364d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:39.148000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107d738-0ec3-45e7-83ca-ca2d3e9364d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.148908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107d738-0ec3-45e7-83ca-ca2d3e9364d5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.154513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.155972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.176104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 287/572] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:39.176104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0922c5c-fe13-4afe-9ed2-464c3edd31ec None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.176104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0922c5c-fe13-4afe-9ed2-464c3edd31ec None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.176104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0922c5c-fe13-4afe-9ed2-464c3edd31ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:39.176567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:39.177517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:39.178478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:39.178478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:39.178478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:39.178478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:39.178478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 286/573] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:39.178478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.198057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c0922c5c-fe13-4afe-9ed2-464c3edd31ec None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:39.199664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c0922c5c-fe13-4afe-9ed2-464c3edd31ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:39.272329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 288/574] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:39.273191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0922c5c-fe13-4afe-9ed2-464c3edd31ec None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.295993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820a4a5-0da3-4587-be60-48f2437c36ec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.315869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.334200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.335004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.335451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.335924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.336988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.347294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.348816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:39.367293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:39.368209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:39.369147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:39.369147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:39.369147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:39.369147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:39.372495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 287/575] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:39.374090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.388070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0922c5c-fe13-4afe-9ed2-464c3edd31ec None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.423685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.424502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.424783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.425057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.436177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.437674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:39.455465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:39.456605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:39.458061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:39.458061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:39.458061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:39.458061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:39.460709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 289/576] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:39.462261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.486149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda9b29-a48c-4353-a392-acd121ee3545 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.523236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.523906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.524185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.524487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.525201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:39.525497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.526894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.539935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 288/577] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:39.540788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.571505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.589001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadadcc7-02ee-4c93-8cb9-2a15b2d50d80 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.601075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.627991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.628860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.629125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.629400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.640170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:39.648244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:39.649512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:39.650366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 290/578] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:39.651088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.664824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd39ec-7b6e-43b1-a9fe-5015eb3ae9db None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.701255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.701927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.702207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.702478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.703194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:39.703495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.704887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.717263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 289/579] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:39.717960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.766120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d325a2-3131-4392-8b52-484384431abc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.803755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.804445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.804727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.804999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.805725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:39.806029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.807390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.816011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.824567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 291/580] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:39.824567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.831708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a99030-2f7a-46a5-be15-4c6d311356d5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.852053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:39.870159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.871098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.871475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.871846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.880019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.881582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:39.896292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:39.897397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:45:39.898483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:39.900579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 290/581] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:39.901493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:39.937663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ea42a-cf9b-4867-9b7c-548fb056f8ee None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:39.982605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:39.983273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:39.983558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:39.983836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:39.984681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:39.984988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:39.986360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:39.995034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 292/582] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:39.995811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.016174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b60a2-d915-42bc-95a5-b66bfc3c96cd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.053251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.054356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.054955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.055418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.063477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.065123 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:40.080203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:40.081222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:40.082688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:40.084804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 291/583] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:40.085964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.093439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:40.103312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:40.109385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4c3eb-4181-4f67-a70a-b6410eaaa335 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.152381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.153199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.153477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.153748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.161145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.162808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:40.178033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:40.178982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:40.179866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 293/584] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:40.181159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.210470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f052d0-cca3-4a53-9863-4db8f1cb1ef7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.250684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.251373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.251893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.252554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.254636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:40.254961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.256581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.284228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 292/585] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:40.284228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.298621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de464e-cc4c-42df-9683-1a4dd3893b2a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.337750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.338501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.338745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.339020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.350454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.352093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.364733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:40.370250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:40.371138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:40.372003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:40.372003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:40.372003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:40.372003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:40.372003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 294/586] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:40.372721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.375332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:40.403984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eafa6-8a2f-4271-ab3f-1085f3d438c5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.440550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.441224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.441654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.441979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.442847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:40.443160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.444610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.464207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 293/587] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:40.469084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.482315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58026713-e544-4643-9295-5ba5995c8ffa None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.518747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.519434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.519748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.520028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.520899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:40.521212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.522665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.540616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 295/588] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:40.540616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.581335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f888b-e77a-4da1-a588-651c34de8436 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.611016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:40.622342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:40.631881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.632723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.633004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.633298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.643878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.645897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.655887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab56093-73c0-4ad6-82e2-4bd43ccebb0a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:40.664851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:40.665743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:40.666688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:40.666688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:40.666688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:40.666688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:40.666688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 294/589] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:40.666977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.693858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.694520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.694803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.695079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.695802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:40.696107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.697513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.709756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 296/590] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:40.710536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.793246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22569a55-ab00-441f-be05-b248af25c18c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.820595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002214-900c-4324-9087-46bf04d6a0dd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:40.828712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.829510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.829791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.830063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.843965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.845468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.846550 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:40.861084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:40.861908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:40.862188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:40.862462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/ac0206e09e4f426d9f4b54543d89c039/roles/94bbdda12f68411882dcceaf22124ca4` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:40.866098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:40.866985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:40.867871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:40.867871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:40.867871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:40.867871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:40.867871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 295/591] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:40.868156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.880839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=ac0206e09e4f426d9f4b54543d89c039, role_id=94bbdda12f68411882dcceaf22124ca4)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:40.882584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:40.897568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 297/592] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:45:40 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/ac0206e09e4f426d9f4b54543d89c039/roles/94bbdda12f68411882dcceaf22124ca4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:40.898366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:40.919907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.022283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2cec0-3d4a-46b7-878f-c9333cf8ccd3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.037293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78ff1-45a8-4be4-96f2-0947b1b773c1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.062220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.062987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.063333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.063607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.064469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:41.064774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.066178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.080479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 296/593] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:41.081440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.089469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.090632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.091118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.092229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.103497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.105157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:41.121176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:41.122317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 15:45:41.123284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:41.123816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 298/594] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:41.124737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.165716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.210381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.221483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65880ee9-af26-4e90-98ee-56b14c5e4b80 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.239259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf9c23-bdf3-4ef8-8316-03e4880382c4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.264025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.264748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.265027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.265307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.266027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:41.266332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.267690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.281107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 297/595] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:41.281740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.285666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c74cc-7783-4db3-9389-b26fbad2a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.287072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c74cc-7783-4db3-9389-b26fbad2a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.287823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c74cc-7783-4db3-9389-b26fbad2a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.289123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c74cc-7783-4db3-9389-b26fbad2a4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.297906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c74cc-7783-4db3-9389-b26fbad2a4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.299775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c74cc-7783-4db3-9389-b26fbad2a4e5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-da5c74cc-7783-4db3-9389-b26fbad2a4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:41.316068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:41.317489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:41.318522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:41.320371 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 299/596] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:41.321751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8eba8-479d-4b2f-a62c-2da2be9a5a16 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.322173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8eba8-479d-4b2f-a62c-2da2be9a5a16 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.322578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8eba8-479d-4b2f-a62c-2da2be9a5a16 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.324171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 300/597] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:41.325014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8eba8-479d-4b2f-a62c-2da2be9a5a16 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.388062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed545489-c815-4401-95ae-5257fd942c36 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.424250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.424595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.425267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.426406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.426406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.427242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:41.427683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.428992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.435500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8eba8-479d-4b2f-a62c-2da2be9a5a16 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.452957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 298/598] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:41.452957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.461179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.480432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c36e6-a1c6-4891-a9b6-c5d813a5a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.481229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c36e6-a1c6-4891-a9b6-c5d813a5a492 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.481506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c36e6-a1c6-4891-a9b6-c5d813a5a492 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.481776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c36e6-a1c6-4891-a9b6-c5d813a5a492 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.490570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c36e6-a1c6-4891-a9b6-c5d813a5a492 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.492017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c36e6-a1c6-4891-a9b6-c5d813a5a492 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fd6c36e6-a1c6-4891-a9b6-c5d813a5a492 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:41.507481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:41.508532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:41.509432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 301/599] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:41.509914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae57e-28b0-44de-a4af-e63ceb2a97a5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.510301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae57e-28b0-44de-a4af-e63ceb2a97a5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.510442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae57e-28b0-44de-a4af-e63ceb2a97a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.527886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4b6ae57e-28b0-44de-a4af-e63ceb2a97a5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:41.529148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4b6ae57e-28b0-44de-a4af-e63ceb2a97a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:41.562346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcd546-ee6f-4f4c-8ba0-e966e14d538d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.604539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.605455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 302/600] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:41.606088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.606287 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae57e-28b0-44de-a4af-e63ceb2a97a5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.606669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.607146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.608069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:41.608551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.609990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.628185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 299/601] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:41.630178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.695220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.708027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.715823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae57e-28b0-44de-a4af-e63ceb2a97a5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.747017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb88b89-10d8-487e-9571-d864ea380378 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.755659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.756485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.756771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.757044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.767921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.769416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.783705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.784393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.784688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.785777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.786383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:41.786573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.787978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:41.788776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:41.789612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:41.790507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:41.790507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:41.790507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:41.790507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:41.790507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 303/602] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:41.790787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.810828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 300/603] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:41.810828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.901275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd9693-e463-4871-83ea-e49bb10f55f3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.918655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14c2de-99fd-4b0b-a9e9-331b77ef4257 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:41.937567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.938453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.938732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.939002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/a31ce1720fa84c43aec2ea016371f475/roles/94bbdda12f68411882dcceaf22124ca4` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.941767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.948901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:41.955239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=a31ce1720fa84c43aec2ea016371f475, role_id=94bbdda12f68411882dcceaf22124ca4)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:41.956700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:41.956990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:41.957866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:41.958146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:41.958413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:41.969548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:41.984793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 304/604] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:45:41 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/a31ce1720fa84c43aec2ea016371f475/roles/94bbdda12f68411882dcceaf22124ca4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:41.984793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:41.985087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:41.985965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 301/605] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:41.986814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:42.108725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f7263-abbe-48dc-8a05-790f59860bc6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:42.140621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44124d-4229-499e-8107-08ee87afaffb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:42.157002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:42.157963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:42.158395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:42.158833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:42.159721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:42.160760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:42.162503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:42.181354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 305/606] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:42.182333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:42.183743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:42.185301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:42.187397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:42.187615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/0b352e6a51fc4d1f8b9b97d092d11bd7/roles/94bbdda12f68411882dcceaf22124ca4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:42.207699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=0b352e6a51fc4d1f8b9b97d092d11bd7, role_id=94bbdda12f68411882dcceaf22124ca4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:42.209082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:42.231519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 302/607] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:45:41 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/0b352e6a51fc4d1f8b9b97d092d11bd7/roles/94bbdda12f68411882dcceaf22124ca4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:42.232417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:42.244839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:42.283304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:42.321111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9a93f-1c1d-4642-9454-9d6889e0395c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:42.363671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:42.364956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:42.365440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:42.365952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:42.366935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:42.367456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:42.369126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:42.384118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 306/608] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:42.391669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:42.435884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09bae5-ef02-4c32-b2c0-05227ce42a3f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:42.503653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:42.503653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:42.503653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:42.504072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:42.514687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:42.516610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:42.533542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:42.534553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:42.537927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 303/609] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:42.540742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:42.577853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a417d-e6f2-46b6-958a-7990b1968363 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:42.588763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:42.629229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:42.632329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:42.632851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:42.633359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:42.656711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:42.658584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:42.696587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:42.697532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:42.698396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:42.698396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:42.698396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:42.698396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:42.707870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 307/610] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:42.718200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6bf49-c9bb-44d1-9008-10832a705560 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:42.723927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:42.724802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:42.796764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:42.797509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:42.797881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:42.798250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:42.799133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:42.799558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:42.801742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:42.827632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 304/611] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:42.829880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:42.892081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21340d2-0da2-422f-bc16-9c44bfd12178 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:42.936202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:42.936975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:42.937254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:42.937522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:42.939058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:42.939368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:42.940829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:42.964436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 308/612] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:42.965858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:43.001943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:43.033735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca588fb0-dd3e-4f17-8ad9-3f53d58d8a58 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:43.069925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:43.070734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.071012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.071288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/dba9f1fa257a4f44ae159850f9ba349c/roles/94bbdda12f68411882dcceaf22124ca4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.102478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=dba9f1fa257a4f44ae159850f9ba349c, role_id=94bbdda12f68411882dcceaf22124ca4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:43.103922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:43.120736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c478718-f54b-4c8b-bc2a-679e32f9ac76 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:43.134868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 305/613] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:45:42 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/dba9f1fa257a4f44ae159850f9ba349c/roles/94bbdda12f68411882dcceaf22124ca4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:45:43.197763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda869b-ba03-4654-8019-73aef8d8212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:43.198602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda869b-ba03-4654-8019-73aef8d8212f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.198887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda869b-ba03-4654-8019-73aef8d8212f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.199181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda869b-ba03-4654-8019-73aef8d8212f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.213283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda869b-ba03-4654-8019-73aef8d8212f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:43.214824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda869b-ba03-4654-8019-73aef8d8212f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-7bda869b-ba03-4654-8019-73aef8d8212f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:43.237021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:43.238059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:43.239979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 309/614] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:43.246751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:43.267277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:43.323507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c1ac7-59ad-4b81-a8d9-b9a9cad21a75 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.324120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c1ac7-59ad-4b81-a8d9-b9a9cad21a75 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.324669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c1ac7-59ad-4b81-a8d9-b9a9cad21a75 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.327815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 310/615] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:43.334224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e634-e1be-47ec-b61a-d7477c3029c6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.334841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e634-e1be-47ec-b61a-d7477c3029c6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.335513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e634-e1be-47ec-b61a-d7477c3029c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.356475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4ba8e634-e1be-47ec-b61a-d7477c3029c6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:43.358862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4ba8e634-e1be-47ec-b61a-d7477c3029c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:43.438723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebc06e-d8a9-4d54-836d-57551d3086cb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:43.451769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 311/616] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:43.478336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8e634-e1be-47ec-b61a-d7477c3029c6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:43.479854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:43.480858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.481143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.481415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.482254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:43.482561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:43.483996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:43.498738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:43.507359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 306/617] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Jun 7 15:45:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:43.513083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:43.594216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:43.634486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe09ed-05d7-466b-b262-7659a51d886f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:43.675641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:43.676645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.676930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.677207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.689264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:43.690745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:43.710905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:43.711843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:43.713026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:43.713026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:43.713026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:43.713026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:43.713026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 307/618] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:43.719061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:43.734210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8e634-e1be-47ec-b61a-d7477c3029c6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:43.782208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:43.783453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.783876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.784290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.804385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:43.814410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:43.815448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:43.816421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:43.820694 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 312/619] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:43.825803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:43.888330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f160a1-4372-40c4-a131-8be0ff9c3d95 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:43.926286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:43.964279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:43.967086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:43.967582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:43.968051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:43.970184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:43.970846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:43.973980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:43.980713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:44.003394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 308/620] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:44.017819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:44.037460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545a20a-1e94-4f96-bd2f-d92ac1092d4b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:44.073409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:44.074228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.074603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.074881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.075629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:44.075930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:44.077366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:44.091148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 313/621] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:44.092532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:44.199523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503a47e-8891-4ad1-a0bb-a1e324f67c2f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:44.201174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0beed1-2a6f-42b7-aa45-9dd73f0dac39 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:44.235627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:44.242687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:44.244426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.244712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.245925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.268908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:44.270879 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:44.272404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:44.273089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.273373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.273695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.286502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:44.287944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:44.304387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:44.305359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:44.306245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:44.310300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 309/622] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:44.311473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:44.312432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:44.313307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:44.314209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:44.314209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:44.314209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:44.314209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:44.320884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 314/623] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:44.337430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:44.366372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:44.432119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b246-9396-479a-92f1-e349e10ff23d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:44.468567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:44.469239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.469524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.469798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.470686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:44.470994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:44.472645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:44.490381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 310/624] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:44.491658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:44.583247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431c9f4-f42e-4792-9408-c0acac4bda73 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:44.596738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:44.601074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df155d0a-2cd6-4c26-82b7-f2a5145a12cd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:44.637026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:44.639185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.639463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.639736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/31137a8197ab4072b47646b893a9733a/roles/94bbdda12f68411882dcceaf22124ca4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.657026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=31137a8197ab4072b47646b893a9733a, role_id=94bbdda12f68411882dcceaf22124ca4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:44.657239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef749032-5f59-4b08-9f4a-f5ceb9979ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:44.657745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749032-5f59-4b08-9f4a-f5ceb9979ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.658030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749032-5f59-4b08-9f4a-f5ceb9979ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.658310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749032-5f59-4b08-9f4a-f5ceb9979ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.658610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:44.659041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749032-5f59-4b08-9f4a-f5ceb9979ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:44.659349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749032-5f59-4b08-9f4a-f5ceb9979ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:44.660860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749032-5f59-4b08-9f4a-f5ceb9979ec7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:44.688653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 311/625] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:45:44 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/31137a8197ab4072b47646b893a9733a/roles/94bbdda12f68411882dcceaf22124ca4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:45:44.693998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 315/626] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:44.699815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:44.724923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bc8c-430f-47d2-b1d1-2abf4f336023 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.725241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bc8c-430f-47d2-b1d1-2abf4f336023 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.725482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bc8c-430f-47d2-b1d1-2abf4f336023 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.727507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 316/627] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:45:44.734226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a3b57-32ba-439d-b350-d01a3497b6a0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.734526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a3b57-32ba-439d-b350-d01a3497b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.734778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a3b57-32ba-439d-b350-d01a3497b6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.765520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-de0a3b57-32ba-439d-b350-d01a3497b6a0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:44.767139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-de0a3b57-32ba-439d-b350-d01a3497b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:44.841992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14209a-cfd0-4d32-b339-255b62826fda None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:44.860746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:44.883116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:44.883947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:44.884300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:44.884667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:44.894810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:44.898190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:44.916925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:44.917983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 15:45:44.919026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:44.923895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 312/628] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:44.925718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:44.938788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:44.950443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 317/629] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 15:45:44.991217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a3b57-32ba-439d-b350-d01a3497b6a0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.092300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315916d-5990-4c98-b5e6-fa49b8202ebd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.179684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a3b57-32ba-439d-b350-d01a3497b6a0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.182860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:45.183798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.184296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.184814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.185924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:45.186577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:45.189122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:45.213687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 313/630] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Jun 7 15:45:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:45.219341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.247937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b2a93-a8c0-459a-85f3-01bc8fffe6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:45.248772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2a93-a8c0-459a-85f3-01bc8fffe6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.249049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2a93-a8c0-459a-85f3-01bc8fffe6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.249317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2a93-a8c0-459a-85f3-01bc8fffe6e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.260763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b2a93-a8c0-459a-85f3-01bc8fffe6e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-650b2a93-a8c0-459a-85f3-01bc8fffe6e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:45.268519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:45.269526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:45.270468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 318/631] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:45.271770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9940f83-6bab-4f1a-bd12-e8794d6e08d8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.272052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9940f83-6bab-4f1a-bd12-e8794d6e08d8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.272323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9940f83-6bab-4f1a-bd12-e8794d6e08d8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.273940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 319/632] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:45.275439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9940f83-6bab-4f1a-bd12-e8794d6e08d8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.309117 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:45.320765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:45.361320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481739ab-b350-430d-9367-c869d0033be9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.391644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9940f83-6bab-4f1a-bd12-e8794d6e08d8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.411022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:45.412303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.412762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.413215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.437179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471d135-9ab7-40eb-96c8-35f88fd56ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:45.438133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471d135-9ab7-40eb-96c8-35f88fd56ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.439097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471d135-9ab7-40eb-96c8-35f88fd56ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.439610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471d135-9ab7-40eb-96c8-35f88fd56ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.440817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471d135-9ab7-40eb-96c8-35f88fd56ea5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:45.441691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471d135-9ab7-40eb-96c8-35f88fd56ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:45.443810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471d135-9ab7-40eb-96c8-35f88fd56ea5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:45.456068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 320/633] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:45.458207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a67a3-71eb-4c56-839f-0da53c29d3a0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.458812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a67a3-71eb-4c56-839f-0da53c29d3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.459233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a67a3-71eb-4c56-839f-0da53c29d3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.484677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f45a67a3-71eb-4c56-839f-0da53c29d3a0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:45.485742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f45a67a3-71eb-4c56-839f-0da53c29d3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:45.503276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:45.506196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:45.568379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 321/634] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:45.568379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a67a3-71eb-4c56-839f-0da53c29d3a0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.590713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:45.609551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:45.657276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 314/635] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 15:45:45.658587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.680630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a67a3-71eb-4c56-839f-0da53c29d3a0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.718179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:45.719168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.719302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.719565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.732235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:45.734617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:45.757819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:45.758945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:45.760011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:45.760011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:45.760011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:45.760011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:45.766335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 322/636] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:45.766335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.780892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9eb82-64c3-4daa-b5ab-6c4a845ec41d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.818585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:45.819382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.819665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.819936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.830255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:45.838224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:45.839170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:45.840006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 315/637] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:45.840723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.866202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:45.872479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:45.893344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c30ec0-7155-4a86-806d-a116c2553f88 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.931602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c0f2c-6a72-4fd0-84f8-ad87510924f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:45.932423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0f2c-6a72-4fd0-84f8-ad87510924f7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.932710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0f2c-6a72-4fd0-84f8-ad87510924f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.932980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c0f2c-6a72-4fd0-84f8-ad87510924f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.945216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0f2c-6a72-4fd0-84f8-ad87510924f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:45.946686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c0f2c-6a72-4fd0-84f8-ad87510924f7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:45.952003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e4092-4323-4dca-83ca-e710ebf56f47 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-411c0f2c-6a72-4fd0-84f8-ad87510924f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:45.967378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:45.968490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:45.969378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:45.969378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:45.969378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:45.969378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:45.969378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 323/638] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:45.970168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9df77f-39eb-4f80-a6b1-f5f128b472b7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:45.970168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9df77f-39eb-4f80-a6b1-f5f128b472b7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:45.970507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9df77f-39eb-4f80-a6b1-f5f128b472b7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:45.971768 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 324/639] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:45.972280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9df77f-39eb-4f80-a6b1-f5f128b472b7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:45.999470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.000524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.001019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.001388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.002104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:46.002409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.004398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.016992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 316/640] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:46.017809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.086452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9df77f-39eb-4f80-a6b1-f5f128b472b7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.112038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.116344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.126223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ea9a4-3800-4ce3-ae88-3b41de088b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.127191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ea9a4-3800-4ce3-ae88-3b41de088b7d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.127481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ea9a4-3800-4ce3-ae88-3b41de088b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.127998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ea9a4-3800-4ce3-ae88-3b41de088b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.130648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddee70-e902-4a24-ae51-bfb5f1b06e93 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.133043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ea9a4-3800-4ce3-ae88-3b41de088b7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:46.133357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ea9a4-3800-4ce3-ae88-3b41de088b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.134787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ea9a4-3800-4ce3-ae88-3b41de088b7d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.147819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 325/641] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:46.147819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a02f-a5f4-4c04-9497-d954d33d119f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.147819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a02f-a5f4-4c04-9497-d954d33d119f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.147819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a02f-a5f4-4c04-9497-d954d33d119f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.163557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b018a02f-a5f4-4c04-9497-d954d33d119f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:46.164774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b018a02f-a5f4-4c04-9497-d954d33d119f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:46.177211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.178086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.178660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.179080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.180542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:46.180997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.182930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.198127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 317/642] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:46.199096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.233562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 326/643] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:46.234321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018a02f-a5f4-4c04-9497-d954d33d119f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.328110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db47043-3cb5-408f-b935-71a7e598ed43 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.345444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018a02f-a5f4-4c04-9497-d954d33d119f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.364565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.365362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.365644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.365917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.369311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.374383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.375810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.384898 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.385690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.385969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.386242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:46.390799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:46.391776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 318/644] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:46.398747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.401677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.403153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.406077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:46.421197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:46.422168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:46.423218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:46.423218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:46.423218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:46.423218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:46.423218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 327/645] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:46.423218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.505236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe75714-664f-4d82-855e-bc1f648da608 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.532091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e3bee-2b9c-41a5-82e1-83f54728435d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.538759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.539557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.539840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.540119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.546920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.548349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:46.563118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:46.564044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:46.565003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 319/646] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:46.565912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.576852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.577894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.578320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.578750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.589532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:46.598912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:46.600313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:46.601359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:46.603390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 328/647] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:46.604399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.637889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.647263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.685099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f284a-302a-4057-a9cd-cddc0251112e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.717517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e87697-afea-4c0e-a8ce-fd6a38fbef26 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.723183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.724032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.724592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.724944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.725709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:46.726016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.727442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.750124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 320/648] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:46.751003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.753325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.754020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.754306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.754580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.755367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:46.755622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.757018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.769993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 329/649] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:46.770633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.863300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c019b-4349-460f-be27-685149c72894 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.872161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.882395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e24281-ab20-4949-836b-08b00d2bbfe1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:46.888046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:46.901616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.902280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.902562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.902838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.903637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:46.904036 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.905515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.922676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:46.922861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 321/650] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:46.922966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:46.922966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:46.922966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:46.923158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:46.923226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:46.923226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:46.923226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:46.936050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 330/651] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:46.936050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.038200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377a673-e880-47a1-8db3-f0dea0c1e76c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.050526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acad4d-110d-4cc5-8d48-080068fc4f6e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.076980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.077771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.078103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.078533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.085792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.086587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.086871 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.087154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.096460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.098572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:47.113069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:47.114623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 331/652] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:47.120430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.141672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.143998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.163669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.165158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.230874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c512a36-85cd-41cc-a6d7-b00fcd1922ed None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.263725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 322/653] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:47.263996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.264569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.264677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.264786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.265081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.265932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=ac0206e09e4f426d9f4b54543d89c039) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:47.266220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.267635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.273798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 332/654] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/role_assignments?user.id=ac0206e09e4f426d9f4b54543d89c039&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:47.274582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.360219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.365460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.384059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dd80c-4938-48a6-9023-4cfa16869fbe None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.385272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca926e16-bde0-4999-affd-d068f13fea2b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.422878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.423696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.423801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedde93-8891-4d27-a264-2f1a6438c98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.424011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.424251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.424471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedde93-8891-4d27-a264-2f1a6438c98c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.424757 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedde93-8891-4d27-a264-2f1a6438c98c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.425032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedde93-8891-4d27-a264-2f1a6438c98c None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.425928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedde93-8891-4d27-a264-2f1a6438c98c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:47.426248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedde93-8891-4d27-a264-2f1a6438c98c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.427735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedde93-8891-4d27-a264-2f1a6438c98c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.436526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.439210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.451138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 333/655] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:47.451138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb27aa-67b2-4471-b1a9-eab43e83d0a8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.451138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb27aa-67b2-4471-b1a9-eab43e83d0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.451138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb27aa-67b2-4471-b1a9-eab43e83d0a8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.451138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 334/656] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:47.451138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb27aa-67b2-4471-b1a9-eab43e83d0a8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:47.466121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:47.467107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:47.467969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:47.467969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:47.467969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:47.467969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:47.467969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 323/657] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:47.468831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.569220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb27aa-67b2-4471-b1a9-eab43e83d0a8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.589114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb22150-a6da-4296-82b8-3b2ac16389c2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.609286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc948c29-2c41-4f5c-8328-278a32fbcb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.610416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948c29-2c41-4f5c-8328-278a32fbcb80 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.610875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948c29-2c41-4f5c-8328-278a32fbcb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.611319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948c29-2c41-4f5c-8328-278a32fbcb80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.615124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.624872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc948c29-2c41-4f5c-8328-278a32fbcb80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.626290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc948c29-2c41-4f5c-8328-278a32fbcb80 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.628020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.630597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.630880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.631152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.636660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-bc948c29-2c41-4f5c-8328-278a32fbcb80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:47.646634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:47.647679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:47.648953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:47.648953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:47.648953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:47.648953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:47.648953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 335/658] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:47.648953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c074-1de1-41d7-aa1b-d064c8dc486d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.649301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c074-1de1-41d7-aa1b-d064c8dc486d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.649301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18c074-1de1-41d7-aa1b-d064c8dc486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.667169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3e18c074-1de1-41d7-aa1b-d064c8dc486d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:47.668340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3e18c074-1de1-41d7-aa1b-d064c8dc486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:47.717415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.718876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.741284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 336/659] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:47.742145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18c074-1de1-41d7-aa1b-d064c8dc486d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.828335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 324/660] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:47.829538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.870875 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.871374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18c074-1de1-41d7-aa1b-d064c8dc486d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.890537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:47.910617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.911364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.911708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.912049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.912866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:47.913255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.914703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:47.926958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 337/661] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:47.927755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:47.946816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6deee0-71c6-4719-893a-34fc0261187b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:47.983125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:47.983787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:47.984065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:47.984335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:47.985141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:47.985933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:47.987610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.003164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 325/662] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:48.004457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.040829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741869b-cbab-4c77-9ac5-dce2f0d1e51e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.078073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.078951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.079297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.079632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.092207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:48.101493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:48.102605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:48.103545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 338/663] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:48.108281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.110062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.116795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34db55b-0ef9-432b-965e-e643355fb4bb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.159039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.163353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.164266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.164663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.164988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.177491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.178943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:48.196393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:48.197430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:48.198353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:48.198353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:48.198353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:48.198353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:48.202085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 326/664] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:48.202913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.226806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bbc6d-c6ac-40b4-aa3c-a507a1d2d9cd None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.263630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.264451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.264736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.265010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.272923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.274347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:48.289793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:48.290769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:48.291711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 339/665] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:48.292312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.315852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555d66-b62b-4241-bb35-0d282013e1ae None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.355736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.356719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.357140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.357467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.366687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.368260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:48.383649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:48.384756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:48.385858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:48.392022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 327/666] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:48.392022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.403352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.411977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.416180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3052413-2507-49eb-8bae-e58d3bb5401d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.454480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.455560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.455845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.456123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.456918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:48.457221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.458845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.471481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 340/667] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:48.471481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.505941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cde1e-ae67-42ee-a0d1-6adf3f66ac1c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.546023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.546345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.546631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.546909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.547730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:48.547994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.549451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.562341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 328/668] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:48.563147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.586323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042378c8-b8e7-40ea-808a-ff00bad2e07f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.622800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.623569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.623853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.624127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.624992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:48.625302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.626727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.641904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.649915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 341/669] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:48.649915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.668623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.684991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4c37d-7068-4f2b-a74a-8c292a7e1dab None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.722972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.724095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.724846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.725266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.726343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:48.726653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.732149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.747208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 329/670] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:48.748058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.766352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c65292-e7fb-4225-92a1-255538e36663 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.802578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.803633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.803922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.804338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.815544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.817524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:48.836504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:48.837440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:48.838323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:48.838323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:48.838323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:48.838323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:48.838323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 342/671] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:48.838596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.854769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d97259-c67d-4044-bd9b-8ded149984ef None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.882103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.890118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.890908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.891186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.891457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:48.898301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:48.899786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:48.914090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:48.915058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:48.915934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 330/672] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:48.916440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:48.922771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:48.955891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82986b04-36e6-4958-900a-dfb1a3f48634 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:48.990433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:48.991201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:48.991387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:48.991625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.027581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64506fb3-547e-424e-8349-dae4d92f6064 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.062469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.063120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.063406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.063681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.064394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.064513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=a31ce1720fa84c43aec2ea016371f475) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:49.064824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.065838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.066309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.080746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 331/673] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/role_assignments?user.id=a31ce1720fa84c43aec2ea016371f475&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:49.080746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.150786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:49.163201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:49.203720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b39b8-7bc8-4540-b5f5-316e0c4fa1a4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.205964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 343/674] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:49.207939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.254727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.255472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.255783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.256127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.257273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:49.257679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.260168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.280527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 332/675] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:49.281845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.366605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef214136-2125-418a-9b6d-b8b901c42235 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.408996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.410811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.410811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.410811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.411827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:49.411937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.413324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.432460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 344/676] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:49.438345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.438507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a6f02-d595-4393-bfe4-a8399f5e85fb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.453847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:49.475519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:49.483636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.484813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.485293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.485784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.499376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.501500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:49.525149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:49.526096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:49.526960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:49.526960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:49.526960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:49.526960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:49.529622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 333/677] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:49.531050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.561517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fd0be-b999-474b-8611-a0142faed569 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.600472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.601306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.601590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.601868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.612455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.615953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:49.641404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:49.642408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:49.643334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:49.647504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 345/678] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:49.648391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.674457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcb364-e619-460d-af3e-d1721342811e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.713556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.714132 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.714416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.714692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.715511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0b352e6a51fc4d1f8b9b97d092d11bd7) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:49.715824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.717299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.723447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 334/679] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/role_assignments?user.id=0b352e6a51fc4d1f8b9b97d092d11bd7&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:49.724245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.758193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:49.786264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:49.792151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4bfc7-b200-4b3b-bdc1-3261e5692161 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.842332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf518b05-826b-4b14-9482-eee1ee844bfd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:49.843581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9361e6-46f9-4afd-ae79-03a1106d3365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.845823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9361e6-46f9-4afd-ae79-03a1106d3365 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.846398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9361e6-46f9-4afd-ae79-03a1106d3365 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.846883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9361e6-46f9-4afd-ae79-03a1106d3365 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.847959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9361e6-46f9-4afd-ae79-03a1106d3365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:49.848516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9361e6-46f9-4afd-ae79-03a1106d3365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.850420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9361e6-46f9-4afd-ae79-03a1106d3365 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.868119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 346/680] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:49.869968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd871111-c2d1-43c2-83c6-5d76ffd184e8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.870469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd871111-c2d1-43c2-83c6-5d76ffd184e8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.870969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd871111-c2d1-43c2-83c6-5d76ffd184e8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.873311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 347/681] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:49.874470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd871111-c2d1-43c2-83c6-5d76ffd184e8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:49.888183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6f207-6778-4e18-b484-624f69e43fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:49.889006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6f207-6778-4e18-b484-624f69e43fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.889356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6f207-6778-4e18-b484-624f69e43fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.889705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6f207-6778-4e18-b484-624f69e43fe1 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.890679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6f207-6778-4e18-b484-624f69e43fe1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:49.891080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6f207-6778-4e18-b484-624f69e43fe1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:49.892648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6f207-6778-4e18-b484-624f69e43fe1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:49.918752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 335/682] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:49.920409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42941e-072e-47cd-81f3-7a6c72725049 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:49.920779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42941e-072e-47cd-81f3-7a6c72725049 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:49.921121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42941e-072e-47cd-81f3-7a6c72725049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:49.940655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-de42941e-072e-47cd-81f3-7a6c72725049 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:49.942180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-de42941e-072e-47cd-81f3-7a6c72725049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:50.017331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd871111-c2d1-43c2-83c6-5d76ffd184e8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.030273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 336/683] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:50.031847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42941e-072e-47cd-81f3-7a6c72725049 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.055204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:50.063534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:50.065747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:50.066541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:50.067622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:50.074766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:50.076207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:50.091737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:50.092907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:50.093801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 348/684] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:50.094295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.110432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:50.159117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42941e-072e-47cd-81f3-7a6c72725049 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.209355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:50.210424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:50.210715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:50.210998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:50.226572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da5a28-5eea-4909-aaee-dc7f7a691d4f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.283640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:50.285606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:50.286038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:50.286748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:50.300256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:50.300636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:50.302078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:50.302195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:50.322288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:50.326877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:50.327945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:50.328907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:50.328907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:50.328907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:50.328907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:50.330180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 349/685] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:50.332589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.413224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:50.434050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 337/686] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:50.435343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.464968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94e652-df59-477b-bdc8-b738a365b5bc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.502835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:50.504338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:50.504810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:50.505232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:50.506162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:50.506617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:50.508193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:50.525365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 350/687] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:50.526558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.567565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c3dfa-0c66-43cd-a8c2-45e0ff9b7d94 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.589447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:50.614903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:50.615715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:50.616101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:50.616669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:50.617849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:50.618274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:50.620039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:50.637294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 338/688] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:50.638522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.640284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3a81a-3ef0-45ae-8c1f-1b575a8de65d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.672162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:50.689389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:50.690244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:50.690797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:50.691138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:50.707133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:50.709023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:50.731979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:50.732956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:50.734504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:50.734504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:50.734504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:50.734504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:50.734504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 351/689] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:50.734504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.763121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b0447-da86-45e5-aa13-cec4273c809b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.810764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:50.811677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:50.811964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:50.812239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:50.813943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=dba9f1fa257a4f44ae159850f9ba349c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:50.814264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:50.815697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:50.832470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 339/690] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:50 2022] GET /identity/v3/role_assignments?user.id=dba9f1fa257a4f44ae159850f9ba349c&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:50.832470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:50.891453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cbda2-1fa6-42e0-b1bb-d090334eb797 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:50.894040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:50.969055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd800d4a-dfcf-4dde-9461-4fa461c2767b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:51.009952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:51.010632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.010913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.011184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.011910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:51.012217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:51.013108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:51.013647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:51.013823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.014084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.014362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.015088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:51.016578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:51.027603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 340/691] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:51.032688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:51.053283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 352/692] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:45:50 2022] POST /identity/v3/users => generated 254 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:51.157965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072030d-80dc-4b52-8b5e-ed2e9add92cd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:51.168738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:51.201046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:51.202009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.202324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.202668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.211656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:51.213288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:51.229629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:51.230886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:51.232004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:51.233402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 341/693] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:51.238749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:51.287136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:51.375282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876a950-64b3-4082-8453-21cf6870ed94 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:51.413815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31295e2b-35d1-47bb-8095-b8ad4505e8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:51.414709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31295e2b-35d1-47bb-8095-b8ad4505e8cb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.415133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31295e2b-35d1-47bb-8095-b8ad4505e8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.415544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31295e2b-35d1-47bb-8095-b8ad4505e8cb None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.416513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31295e2b-35d1-47bb-8095-b8ad4505e8cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:51.416958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31295e2b-35d1-47bb-8095-b8ad4505e8cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:51.418559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31295e2b-35d1-47bb-8095-b8ad4505e8cb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:51.433117 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 353/694] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:51.439405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:51.570362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2a824-7ce3-4563-b957-59b8b7374b01 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.570874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2a824-7ce3-4563-b957-59b8b7374b01 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.570874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2a824-7ce3-4563-b957-59b8b7374b01 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.573164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 354/695] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:51.580830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346241d-ee07-47e6-9384-0e3e15e325fa None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.581248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346241d-ee07-47e6-9384-0e3e15e325fa None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.581724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346241d-ee07-47e6-9384-0e3e15e325fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.589843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070b47a-e2f4-42cb-b683-52af58d875c4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:51.612786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f346241d-ee07-47e6-9384-0e3e15e325fa None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:51.614811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f346241d-ee07-47e6-9384-0e3e15e325fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:51.642144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:51.642801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.643075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.643344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.644440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=31137a8197ab4072b47646b893a9733a) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:51.644588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:51.646068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:51.663891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 342/696] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:51 2022] GET /identity/v3/role_assignments?user.id=31137a8197ab4072b47646b893a9733a&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:51.737312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 355/697] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:51.787016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:51.825924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:51.929366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964d216-bc48-406f-9fca-6040e4a17802 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:51.966967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:51.967786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:51.968071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:51.968369 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:51.969032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:51.970418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:51.985948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:51.986956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:45:51.987797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:51.989534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 343/698] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Jun 7 15:45:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:51.994069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346241d-ee07-47e6-9384-0e3e15e325fa None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:52.017444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:52.104829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346241d-ee07-47e6-9384-0e3e15e325fa None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:52.154152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562d563-346a-49e3-adfd-835d8b2dac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:52.154841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562d563-346a-49e3-adfd-835d8b2dac89 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:52.155122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562d563-346a-49e3-adfd-835d8b2dac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:52.155396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562d563-346a-49e3-adfd-835d8b2dac89 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:52.156114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562d563-346a-49e3-adfd-835d8b2dac89 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:52.156447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562d563-346a-49e3-adfd-835d8b2dac89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:52.157823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562d563-346a-49e3-adfd-835d8b2dac89 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:52.168218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 356/699] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Jun 7 15:45:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:52.169669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:52.288029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:52.304746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c5e1-f72a-442e-a891-08a01ea83dc8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:52.347048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bea603-1b53-4b14-b805-c69b263cc09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:52.347842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bea603-1b53-4b14-b805-c69b263cc09b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:52.348257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bea603-1b53-4b14-b805-c69b263cc09b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:52.348578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bea603-1b53-4b14-b805-c69b263cc09b None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:52.349601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bea603-1b53-4b14-b805-c69b263cc09b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:52.349912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bea603-1b53-4b14-b805-c69b263cc09b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:52.351289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bea603-1b53-4b14-b805-c69b263cc09b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:52.360467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 344/700] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Jun 7 15:45:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:52.513721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4cb05-3c31-47a1-b83e-69c5f761f384 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:52.514101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4cb05-3c31-47a1-b83e-69c5f761f384 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:52.514282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4cb05-3c31-47a1-b83e-69c5f761f384 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:52.515992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 357/701] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:52.522213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d431e62-067c-4159-aaac-2cf460cee010 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:52.522653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d431e62-067c-4159-aaac-2cf460cee010 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:52.523066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d431e62-067c-4159-aaac-2cf460cee010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:52.553953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4d431e62-067c-4159-aaac-2cf460cee010 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:52.555094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4d431e62-067c-4159-aaac-2cf460cee010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:52.634062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 345/702] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:52.660281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4cb05-3c31-47a1-b83e-69c5f761f384 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:52.799755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:52.824260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4cb05-3c31-47a1-b83e-69c5f761f384 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:52.873140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:52.873990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:52.874322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:52.874602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:52.885582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:52.893333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:52.894793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:52.895762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:52.896455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 358/703] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:52.903232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d431e62-067c-4159-aaac-2cf460cee010 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:53.023073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:53.051693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d431e62-067c-4159-aaac-2cf460cee010 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:53.089258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:53.089849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:53.090127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:53.090396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:53.091131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:53.091431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:53.092818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:53.102501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 346/704] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:53.107614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:53.257593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40caab-8f2d-451a-b322-2eeac718bda0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:53.297542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:53.298425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:53.298712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:53.298989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:53.316902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:53.318372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:53.321232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:53.341218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:53.342133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:53.343017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:53.343017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:53.343017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:53.343017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:53.343017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 359/705] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:53.348843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:53.468332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbfd0c-906f-4ab9-ba40-892d7574d8e3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:53.507496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:53.508174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:53.508488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:53.508763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:53.509487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:53.509789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:53.511175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:53.524214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 347/706] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:53.529254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:53.739029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725990d9-f1b8-47d8-ac1f-b30d2fcc1de4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:53.785610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4faed6-6823-4808-b4ec-1c24c7fa21f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:53.786411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4faed6-6823-4808-b4ec-1c24c7fa21f7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:53.786686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4faed6-6823-4808-b4ec-1c24c7fa21f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:53.786954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4faed6-6823-4808-b4ec-1c24c7fa21f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:53.798634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4faed6-6823-4808-b4ec-1c24c7fa21f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:53.800069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4faed6-6823-4808-b4ec-1c24c7fa21f7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2c4faed6-6823-4808-b4ec-1c24c7fa21f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:53.821372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:53.822361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:53.823292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 360/707] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:45:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:53.828014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:53.832880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:53.938994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5770d-2180-4637-a014-e4f1ba80fc22 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:53.944197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f793-13c8-4262-b16c-c0d031d512e4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:53.944514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f793-13c8-4262-b16c-c0d031d512e4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:53.944792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f793-13c8-4262-b16c-c0d031d512e4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:53.946145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 361/708] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:53.951424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590510-48a5-4ac5-a35b-0f984c78d5b9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:53.951700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590510-48a5-4ac5-a35b-0f984c78d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:53.951973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590510-48a5-4ac5-a35b-0f984c78d5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:53.975674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b5590510-48a5-4ac5-a35b-0f984c78d5b9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:53.976876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b5590510-48a5-4ac5-a35b-0f984c78d5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:54.000292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.000922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.001202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.001477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.002303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:54.002608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.004908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:54.023126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 348/709] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:45:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:54.033474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.040807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.080466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 362/710] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:54.083158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf484d-5023-4dc2-9e4a-5681b124ddef None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.083559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf484d-5023-4dc2-9e4a-5681b124ddef None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.084060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf484d-5023-4dc2-9e4a-5681b124ddef None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.085827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 363/711] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:54.090912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e981ada-5e53-4a04-9674-69b5743482fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.091198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e981ada-5e53-4a04-9674-69b5743482fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.091475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e981ada-5e53-4a04-9674-69b5743482fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.114205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0e981ada-5e53-4a04-9674-69b5743482fe None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:54.115719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0e981ada-5e53-4a04-9674-69b5743482fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:54.119048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.163952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215e92b-a9e5-49fe-925a-a4ea0d2ffc96 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.203910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.204751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.205099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.205449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.215758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 364/712] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 15:45:54.216931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e981ada-5e53-4a04-9674-69b5743482fe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.286950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.288322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:54.343571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e981ada-5e53-4a04-9674-69b5743482fe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.361636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.365066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.385366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.386169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.386451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.386726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] PATH_INFO: `/v3/services/network` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.387711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.388888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:54.407208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:54.408304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:45:54.409189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:54.413603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 365/713] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:54.414607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.417699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 349/714] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:54.419612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.530892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a52da5-9416-4772-b5e3-4f3641102474 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.546020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bbb7b-c23c-4d3a-80bf-badb992144ec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.569339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.570149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.570434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.570713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.582022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:54.588615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.589552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.590044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.590543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:54.590722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:54.591944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:45:54.592993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:54.593658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] RBAC: Adding query filter params (name=network) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:54.593658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.594911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:54.596006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 366/715] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:54.597803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.605045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 350/716] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:54.605045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.636511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.656183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.727203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d93f6-9f85-4b9f-887e-493018c8d091 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.732386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52c7aa-f575-4788-9cd4-843996f8ac4f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.766065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.767569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.768169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.768767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.773945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.774612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.774889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.775164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.775896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:54.776197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.777585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:54.782962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.785370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:54.788038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 351/717] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:54.789219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:54.805888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:54.806839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:54.807778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:54.807778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:54.807778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:54.807778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:54.810407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 367/718] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:54.811704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.897999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.913769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f1d2f-19a1-4911-8624-7c0c567a32c5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.931961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:54.942059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57387e12-d36b-4813-b2ab-e0b7e7c759b8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:54.955348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.956329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.957384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.957940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.958977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:54.959586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.961207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:54.972654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 352/719] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:54.974061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:54.982293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd45c6-a47c-4416-aaf4-26073b1fa0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:54.983126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd45c6-a47c-4416-aaf4-26073b1fa0ef None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:54.983410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd45c6-a47c-4416-aaf4-26073b1fa0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:54.983681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd45c6-a47c-4416-aaf4-26073b1fa0ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:54.996918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd45c6-a47c-4416-aaf4-26073b1fa0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:54.998929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd45c6-a47c-4416-aaf4-26073b1fa0ef None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-99cd45c6-a47c-4416-aaf4-26073b1fa0ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:55.023647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:55.024667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:55.025558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:55.025558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:55.025558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:55.025558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:55.026555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 368/720] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:55.027314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a239-c785-4377-b766-655d3d3739fa None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.027568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a239-c785-4377-b766-655d3d3739fa None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.027843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a239-c785-4377-b766-655d3d3739fa None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.029475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 369/721] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:55.031767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a239-c785-4377-b766-655d3d3739fa None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.110966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158330-8ffc-4d93-89be-130f29ad0ef0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.149233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e93f7-81a9-4c8f-8894-4bb30a2e2e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.149921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e93f7-81a9-4c8f-8894-4bb30a2e2e9c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.150530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e93f7-81a9-4c8f-8894-4bb30a2e2e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.150530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e93f7-81a9-4c8f-8894-4bb30a2e2e9c None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.151179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e93f7-81a9-4c8f-8894-4bb30a2e2e9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:55.151484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e93f7-81a9-4c8f-8894-4bb30a2e2e9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:55.152897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e93f7-81a9-4c8f-8894-4bb30a2e2e9c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.166264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 353/722] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:55.168024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a392b-28a8-4e7f-beda-302d129ae0c4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.168457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a392b-28a8-4e7f-beda-302d129ae0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.168834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a392b-28a8-4e7f-beda-302d129ae0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.181135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a239-c785-4377-b766-655d3d3739fa None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.187239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9a9a392b-28a8-4e7f-beda-302d129ae0c4 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:55.188583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9a9a392b-28a8-4e7f-beda-302d129ae0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:55.198882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:55.222724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.223389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.223838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.223960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.224933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:55.225231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:55.226673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.240261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 370/723] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:55.241134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.248730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:55.268520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 354/724] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:55.269946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82cdbf-317b-4a57-965e-fbb47de7f466 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.270228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82cdbf-317b-4a57-965e-fbb47de7f466 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.270501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82cdbf-317b-4a57-965e-fbb47de7f466 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.272025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 355/725] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:55.276557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82cdbf-317b-4a57-965e-fbb47de7f466 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.365834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef045-6914-49b3-a625-feb24097b427 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.393047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82cdbf-317b-4a57-965e-fbb47de7f466 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.404330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c7d91-d452-48d1-a395-57af7c28896b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.405264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7d91-d452-48d1-a395-57af7c28896b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.405643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7d91-d452-48d1-a395-57af7c28896b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.406003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7d91-d452-48d1-a395-57af7c28896b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.414530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c7d91-d452-48d1-a395-57af7c28896b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:55.416011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c7d91-d452-48d1-a395-57af7c28896b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.427560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.428390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.428676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.428950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-748c7d91-d452-48d1-a395-57af7c28896b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:55.430107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:55.431024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 371/726] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4436c-3200-4d99-9606-1525ff64265c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.436690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4436c-3200-4d99-9606-1525ff64265c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.437385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4436c-3200-4d99-9606-1525ff64265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.444428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:55.446045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.452255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-23b4436c-3200-4d99-9606-1525ff64265c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:55.453476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-23b4436c-3200-4d99-9606-1525ff64265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:55.463284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:55.464443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:45:55.465566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:55.466447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 356/727] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:55.467228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.469999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:55.522584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:55.531828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 372/728] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:55.532876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4436c-3200-4d99-9606-1525ff64265c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.588927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193803b-f39f-4848-bff7-65cb39c36521 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.626651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a927-0887-4802-8327-4febeb49232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.627359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a927-0887-4802-8327-4febeb49232a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.627635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a927-0887-4802-8327-4febeb49232a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.628127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a927-0887-4802-8327-4febeb49232a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.639866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760a927-0887-4802-8327-4febeb49232a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:55.645231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4436c-3200-4d99-9606-1525ff64265c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-3760a927-0887-4802-8327-4febeb49232a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:55.648658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:55.649860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 357/729] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:55.650703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a927-0887-4802-8327-4febeb49232a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.690139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.691122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.691676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.692138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.693145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:55.693451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:55.694878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.712863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 373/730] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:55.713827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.722980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:55.765880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a927-0887-4802-8327-4febeb49232a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.775982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:55.808008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.808979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.809260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.809671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.810791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:55.811101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:55.812539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.826258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432afab-c40c-40cf-99b7-71bc7654c93c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:55.837551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 358/731] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:55.837551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.863430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:55.864169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:55.864481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:55.864750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:55.865462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:55.866812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:55.883617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 374/732] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:45:55 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:55.884188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:55.982675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fc05a-ec50-4cbf-906f-3949cc01bcb5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.009108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:56.013301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ecbe9-0e0c-40e7-8f07-4c49938c3c3a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.026218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:56.028780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:56.029729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:56.030238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:56.030711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:56.031738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:56.032306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:56.033975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:56.044957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 359/733] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:56.046169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:56.057852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:56.058530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:56.058815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:56.059094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:56.059943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0b352e6a51fc4d1f8b9b97d092d11bd7) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:56.060251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:56.061854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:56.075206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 375/734] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:55 2022] GET /identity/v3/role_assignments?user.id=0b352e6a51fc4d1f8b9b97d092d11bd7&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:56.075206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:56.218848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eb3bb-bb45-4f69-8809-08ab224e3c9b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.229450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77459536-6ee2-42bf-bd41-46874eaa18fe None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.265458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:56.266120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:56.266399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:56.266698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:56.267831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:56.268720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:56.269083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:56.269358 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:56.282274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:56.283766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:56.303797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:56.304781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:56.305732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:56.305732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:56.305732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:56.305732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:56.305953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 376/735] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:56.311880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:56.329586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:56.345843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:56.364113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:56.366278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:56.422515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277643f-d538-429b-b130-dd123ea054ec None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.458256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:56.459838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:56.460348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:56.460822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:56.461962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:56.462295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:56.463947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:56.476579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 360/736] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:56.480043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 377/737] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:56.485106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:56.485631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:56.579096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:56.591089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b639fe-d000-49f2-9038-8e711699aa36 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.627592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:56.628490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:56.628779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:56.629059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:56.636901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:56.638480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:45:56.654400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:56.655405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:56.656703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 378/738] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:56.661613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:56.771371 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb895f-6b55-47f2-9eb3-1366dd8c8e9d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.808176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:56.808945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:56.809154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:56.809431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:56.810736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:56.811056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:56.812523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:56.832581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 379/739] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Jun 7 15:45:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:56.834004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:56.838456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:56.847958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:56.873037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012c38d-6c60-4d77-9509-7d164776968c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:56.964638 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5f1b0-a12b-4daf-bf55-d705e8396bf6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:57.001843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:57.002546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.002834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.003111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.025171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:57.025906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.026187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.026460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.074741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:57.076198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:57.077255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:57.078709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:57.079165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:57.138732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:57.139720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:57.140691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:57.140691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:57.140691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:57.140691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:57.148788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 361/740] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 664 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:57.157345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:57.187695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 380/741] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:45:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 15:45:57.193221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:57.311331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe6673-4401-4209-8d11-b5181c55e88b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:57.320668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:57.374993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:57.375799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.376081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.376388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.405520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:57.406952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8cdff-9c6f-4c6c-a2d9-6269e6098593 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:57.407071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:57.443526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:57.444207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.444537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.444814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.445537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:57.445843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:57.447216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:57.452409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:57.453290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:57.454587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:57.454587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:57.454587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:57.454587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:57.454587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 381/742] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:57.457211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:57.461575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:57.465479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 362/743] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:57.473011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:57.626439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581303c6-e454-4728-bbf6-0a6f19b3624a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:57.655767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750d045-aadc-4594-bb2b-1ec6a6c5042c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:57.669826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4510c-e114-435a-ad32-72e093b1b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:57.670709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4510c-e114-435a-ad32-72e093b1b38a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.671293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4510c-e114-435a-ad32-72e093b1b38a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.671589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4510c-e114-435a-ad32-72e093b1b38a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.681012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4510c-e114-435a-ad32-72e093b1b38a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:57.682824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4510c-e114-435a-ad32-72e093b1b38a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:57.696997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930cab-1bfa-4219-8139-8f9d0f365f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:57.698113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930cab-1bfa-4219-8139-8f9d0f365f34 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.698580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930cab-1bfa-4219-8139-8f9d0f365f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.699033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930cab-1bfa-4219-8139-8f9d0f365f34 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.699927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930cab-1bfa-4219-8139-8f9d0f365f34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-37a4510c-e114-435a-ad32-72e093b1b38a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:57.700292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:57.701183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:57.702155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:57.703266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930cab-1bfa-4219-8139-8f9d0f365f34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:57.704845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930cab-1bfa-4219-8139-8f9d0f365f34 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:57.706280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 363/744] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:57.708451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf444b-25f4-41f1-b837-f456f7ef4128 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.708741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf444b-25f4-41f1-b837-f456f7ef4128 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.709014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf444b-25f4-41f1-b837-f456f7ef4128 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.710525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 364/745] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:57.711220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf444b-25f4-41f1-b837-f456f7ef4128 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:57.720816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 382/746] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:57.722829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0248e2-e44a-46e8-805d-cfd54cb42703 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.723596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0248e2-e44a-46e8-805d-cfd54cb42703 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.724331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0248e2-e44a-46e8-805d-cfd54cb42703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.745402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cc0248e2-e44a-46e8-805d-cfd54cb42703 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:57.746846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cc0248e2-e44a-46e8-805d-cfd54cb42703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:57.758943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:57.797835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:57.824070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 383/747] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 15:45:57.825551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0248e2-e44a-46e8-805d-cfd54cb42703 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:57.878183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf444b-25f4-41f1-b837-f456f7ef4128 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:57.929636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:57.930821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:57.931156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:57.931441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:57.932231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:57.932965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:57.934770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:57.951161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 365/748] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:57.957548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:57.963628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0248e2-e44a-46e8-805d-cfd54cb42703 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:58.002708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:58.003952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.004504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.005012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.012642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:58.014367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:45:58.029587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:58.030529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:45:58.031415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:58.034066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 384/749] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:58.035205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:58.046230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:58.091263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df623400-ab4d-4a13-ab99-2efe3b3892ee None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:58.110864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:58.140308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:58.141342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.142386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.142896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.144613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:58.147144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:58.159545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d2316-a808-4d91-b20a-6287ef60584c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:58.168499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 366/750] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:45:57 2022] POST /identity/v3/services => generated 240 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:45:58.169630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:58.242605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:58.243288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.243574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.244394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.245563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=d95f13b6e27c43a7868a8ef79621eb7e) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:58.245882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:58.247386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:58.255585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 385/751] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:58 2022] GET /identity/v3/role_assignments?user.id=d95f13b6e27c43a7868a8ef79621eb7e&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:58.366176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865879-56ce-44ff-87bc-570ff0633677 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:58.411210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547e800-fde8-4b40-810e-812b3c2125f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:58.412274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547e800-fde8-4b40-810e-812b3c2125f0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.412797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547e800-fde8-4b40-810e-812b3c2125f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.413304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547e800-fde8-4b40-810e-812b3c2125f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.414339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547e800-fde8-4b40-810e-812b3c2125f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:58.414887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547e800-fde8-4b40-810e-812b3c2125f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:58.416726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547e800-fde8-4b40-810e-812b3c2125f0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:58.440173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 367/752] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:58.446869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:58.546415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:58.586388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467decf-be92-4c12-8858-86ee7ae5e388 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:58.622915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9e830-671a-40df-a270-901c04f80246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:58.623581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9e830-671a-40df-a270-901c04f80246 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.623866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9e830-671a-40df-a270-901c04f80246 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.624145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9e830-671a-40df-a270-901c04f80246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.625009 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9e830-671a-40df-a270-901c04f80246 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=31137a8197ab4072b47646b893a9733a) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:58.625473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9e830-671a-40df-a270-901c04f80246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:58.626835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9e830-671a-40df-a270-901c04f80246 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:58.627029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f659-b606-484e-8702-1f4c1b04d862 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.627446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f659-b606-484e-8702-1f4c1b04d862 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.627615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f659-b606-484e-8702-1f4c1b04d862 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.629263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 368/753] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:45:58.633424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 386/754] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:45:58 2022] GET /identity/v3/role_assignments?user.id=31137a8197ab4072b47646b893a9733a&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:58.634415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064047e1-9a35-4250-b520-4a1dc3f95969 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.634707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064047e1-9a35-4250-b520-4a1dc3f95969 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.634987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064047e1-9a35-4250-b520-4a1dc3f95969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.658039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-064047e1-9a35-4250-b520-4a1dc3f95969 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:58.659268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-064047e1-9a35-4250-b520-4a1dc3f95969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:58.692432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:58.702900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096a2c4-cdca-4145-8329-dd54e21c1824 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.703267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096a2c4-cdca-4145-8329-dd54e21c1824 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.703641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096a2c4-cdca-4145-8329-dd54e21c1824 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.705633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 387/755] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:45:58.713159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e6b07-bca2-4001-bcd5-b22cbfb25a8d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:58.713465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e6b07-bca2-4001-bcd5-b22cbfb25a8d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:58.713857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e6b07-bca2-4001-bcd5-b22cbfb25a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:58.749456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5f4e6b07-bca2-4001-bcd5-b22cbfb25a8d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:45:58.753486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5f4e6b07-bca2-4001-bcd5-b22cbfb25a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:45:58.772258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 369/756] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:45:58.801875 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064047e1-9a35-4250-b520-4a1dc3f95969 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:58.866771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 388/757] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:45:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:45:58.892473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e6b07-bca2-4001-bcd5-b22cbfb25a8d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:58.968729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064047e1-9a35-4250-b520-4a1dc3f95969 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:58.972731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:59.007256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.008076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.008377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.008654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.009307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:59.011352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:59.026903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:59.027887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:45:59.028790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 370/758] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Jun 7 15:45:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:59.037511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:59.077409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e6b07-bca2-4001-bcd5-b22cbfb25a8d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:59.092756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:59.165053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.165918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.166204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.166481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.167327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06395b32-0265-421c-86df-5a8af11e8dd3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:59.197119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:45:59.202293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.202953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.203230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.203498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.204191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:59.204534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:59.205894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:45:59.210551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:59.211522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:45:59.212392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:59.213042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 371/759] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Jun 7 15:45:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:59.215261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 389/760] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:45:59.220450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:59.226140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:59.315208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:59.325202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be00ff-50d2-4ab8-931a-be2893c9229a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:59.360682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.361672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.362135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.363136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.364068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:59.364569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:59.366124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:59.371451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 372/761] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Jun 7 15:45:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:59.377320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:59.486044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefab61-e378-4e94-ac5f-089cdffd3aeb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:59.490845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f5c8-dbeb-4f89-aa4a-7182702c6311 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:59.525486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.526519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.526978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.527425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.528681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] RBAC: Adding query filter params (interface=public, service_id=24fd1449a3604077bb6b6b7114b2cc4b, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:59.529225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:59.531247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:59.541261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 373/762] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:45:59 2022] GET /identity/v3/endpoints?service_id=24fd1449a3604077bb6b6b7114b2cc4b&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:45:59.546762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:59.552039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:59.570417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:45:59.605818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.606509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.606796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.607075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.607832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:45:59.608139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:59.609548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:59.633510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 390/763] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:45:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:59.639355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:45:59.659058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c477-6de2-476d-b5e4-80fa972b0e00 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:59.698166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.699295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.699797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.700282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.701142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:59.702529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:59.709203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 374/764] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:45:59 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:45:59.880175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce358-8c43-4a0b-be97-bb88762e93ee None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:45:59.930668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:45:59.931480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:45:59.931760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:45:59.932035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:45:59.944489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:45:59.945940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:45:59.964200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:45:59.965155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:45:59.966464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:45:59.966464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:45:59.966464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:45:59.966464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:45:59.966464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 391/765] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:45:59.971287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:00.058725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:00.076617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320eabc-abe3-4284-aeb5-b38529497fcf None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:00.113839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:00.114509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:00.114793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:00.115069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:00.115795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:00.116103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:00.117607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:00.139398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 375/766] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:00.147841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:00.200671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:00.292260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10c786-62e7-41e4-870a-5ad60fd8aada None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:00.332098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:00.333046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:00.333331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:00.333664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:00.341534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:00.342994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:00.357804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:00.358818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:00.360379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 392/767] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:46:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:00.365282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:00.507298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc437d-7a59-4331-9e54-fe30776ca497 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:00.559165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:00.559873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:00.560155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:00.560476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:00.561337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:00.561650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:00.563163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:00.583890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 376/768] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:46:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:00.588984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:00.778175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:00.792952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d4373-eaf8-42f5-ac96-9624a43ad4bf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:00.846306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e73ee-d328-4d99-a50e-38c634515bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:00.848524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e73ee-d328-4d99-a50e-38c634515bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:00.848951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e73ee-d328-4d99-a50e-38c634515bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:00.849741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e73ee-d328-4d99-a50e-38c634515bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:00.866229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e73ee-d328-4d99-a50e-38c634515bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:00.867908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e73ee-d328-4d99-a50e-38c634515bf6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6d2e73ee-d328-4d99-a50e-38c634515bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:00.888839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:00.890366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:00.891457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:00.891457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:00.891457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:00.891457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:00.893062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 393/769] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:00.899147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:00.906499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:00.932474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2e31-b964-4e55-8012-f6d1aa46e81d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:00.932765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2e31-b964-4e55-8012-f6d1aa46e81d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:00.933050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2e31-b964-4e55-8012-f6d1aa46e81d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:00.934671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 394/770] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:00.939774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e930ea-d6da-4453-a5df-f6bda46431aa None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:00.940061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e930ea-d6da-4453-a5df-f6bda46431aa None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:00.940339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e930ea-d6da-4453-a5df-f6bda46431aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:00.958419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a7e930ea-d6da-4453-a5df-f6bda46431aa None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:00.959603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a7e930ea-d6da-4453-a5df-f6bda46431aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:01.090494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 395/771] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:01.109916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66dc33-e943-4848-8108-812465f4ca81 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.119064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45700487-b16b-459a-af39-161ffb03872d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.119355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45700487-b16b-459a-af39-161ffb03872d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.119632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45700487-b16b-459a-af39-161ffb03872d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.121290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 396/772] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:01.134485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e9770-2c09-404c-8229-5629159805d9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.134868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e9770-2c09-404c-8229-5629159805d9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.135222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e9770-2c09-404c-8229-5629159805d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.154791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-936e9770-2c09-404c-8229-5629159805d9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:01.155976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-936e9770-2c09-404c-8229-5629159805d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:01.157654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.159356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.159988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.160507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.161391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:01.161837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:01.163350 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:01.179005 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 377/773] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:01.184868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.236504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 397/774] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:01.237739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e9770-2c09-404c-8229-5629159805d9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.256695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:01.272778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:01.320503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1707692-2f7c-434e-b35d-d77ce5949706 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.361270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.361971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.362248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.362431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e9770-2c09-404c-8229-5629159805d9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.362565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.372994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:01.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:01.382193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 378/775] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:01.383263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.405036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1d1cc-1da2-45fd-b167-5f70fb214c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.405831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1d1cc-1da2-45fd-b167-5f70fb214c54 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.406119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1d1cc-1da2-45fd-b167-5f70fb214c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.406395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1d1cc-1da2-45fd-b167-5f70fb214c54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.413391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1d1cc-1da2-45fd-b167-5f70fb214c54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:01.414833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1d1cc-1da2-45fd-b167-5f70fb214c54 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-03a1d1cc-1da2-45fd-b167-5f70fb214c54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:01.428690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:01.429551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 398/776] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc81d1-94ab-42a6-a96b-4a1e0b2a57a4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.435230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc81d1-94ab-42a6-a96b-4a1e0b2a57a4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.435822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc81d1-94ab-42a6-a96b-4a1e0b2a57a4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.435822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 399/777] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:01.435822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc81d1-94ab-42a6-a96b-4a1e0b2a57a4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.502795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe09b2-ae63-4fdf-911d-f84d6507f00b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.509402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:01.528471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:01.542023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.542815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.543098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.543371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] PATH_INFO: `/v3/services/network` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.544017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:01.545421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:01.548299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc81d1-94ab-42a6-a96b-4a1e0b2a57a4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:01.558568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:01.559565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:01.560420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 379/778] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:01.561193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.590085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1041c-fa81-4a50-a0ed-a8fc90b4f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.590700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1041c-fa81-4a50-a0ed-a8fc90b4f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.591046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1041c-fa81-4a50-a0ed-a8fc90b4f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.591487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1041c-fa81-4a50-a0ed-a8fc90b4f5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.592128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1041c-fa81-4a50-a0ed-a8fc90b4f5a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:01.592386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1041c-fa81-4a50-a0ed-a8fc90b4f5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:01.593755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1041c-fa81-4a50-a0ed-a8fc90b4f5a3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:01.608445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 400/779] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:01.608445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4a32-04c9-438e-9336-bff8b3c4bd75 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.608445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4a32-04c9-438e-9336-bff8b3c4bd75 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.608445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4a32-04c9-438e-9336-bff8b3c4bd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.624552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-001b4a32-04c9-438e-9336-bff8b3c4bd75 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:01.625924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-001b4a32-04c9-438e-9336-bff8b3c4bd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:01.672220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6636c4-267f-46a3-85e5-04bef7075807 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.700210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 401/780] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:01.701361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4a32-04c9-438e-9336-bff8b3c4bd75 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.729819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.730484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.730764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.731050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.731818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:01.732124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:01.733579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:01.751584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 380/781] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:01.751584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.778489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:01.806673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:01.823570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4a32-04c9-438e-9336-bff8b3c4bd75 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.865516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.865720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b837746-c2f0-4aa6-9ab5-2eae3ed5445a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:01.866177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.866459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.866729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.867423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:01.867720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:01.869101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:01.873793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 402/782] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:01.874479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.907210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:01.908058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:01.908277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:01.908566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:01.920677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:01.922240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:01.943126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:01.944052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:01.945283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:01.945283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:01.945283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:01.945283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:01.945283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 381/783] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:01.945283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:01.987038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc4351-8960-4f58-9c6b-b13955aa35d2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.013271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.027865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.028078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.028243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.028527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.039478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:02.047502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:02.048668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 403/784] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:02.049548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.061841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.066709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab17cc1-0ab6-41d1-aa20-1645fc0d9d56 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.103386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.104270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.104589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.104869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/0b352e6a51fc4d1f8b9b97d092d11bd7/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.106870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=0b352e6a51fc4d1f8b9b97d092d11bd7, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.108343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.121242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 382/785] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:46:01 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/0b352e6a51fc4d1f8b9b97d092d11bd7/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:46:02.122041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.183769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0bd23-4f07-466e-9edb-fdc562890590 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.226822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.227603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.227986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.228322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.229099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] RBAC: Adding query filter params (type=network) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:02.229408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.230917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.242682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 404/786] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:02.242682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.284182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda152b9-134e-4559-a435-bc79045adc51 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.301378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.346711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.348155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.348462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.348733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.349770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:02.350074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.351498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.367572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 383/787] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:02.368632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.382341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de214244-b6dd-4807-a9af-38fd6c62c7ec None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.396072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.422103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.423517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.423999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.424802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.426151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:02.426659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.428899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.439736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 405/788] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:02.441121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.497149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d54c6c-b02f-41b1-b570-4debf642f057 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.534058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.535121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.535579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.536054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.537032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] RBAC: Adding query filter params (interface=public, service_id=d9557e0b1821427fb58f74b87d625531, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:02.537609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.539237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.549803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 384/789] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/endpoints?service_id=d9557e0b1821427fb58f74b87d625531&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:02.551580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.554909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fabd4d-bae9-4d19-b67f-489425c51679 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.564734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.593438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.594372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.594723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.595074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.604220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.606029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:02.621780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:02.622832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:02.623764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:02.627708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 406/790] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:02.628935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.648289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.670419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bd734-a5ae-4b7c-a318-6d1de22e4f66 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.708939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.709926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.710321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.710609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.722734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.724279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:02.743503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:02.744454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:02.745351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:02.745351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:02.745351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:02.745351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:02.745351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 385/791] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:02.745642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.753664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcf460-0cdf-476b-a98c-092baa00087d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.791479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068ff37-0594-4fa8-b5e4-2867e5a26742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.792140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ff37-0594-4fa8-b5e4-2867e5a26742 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.792446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ff37-0594-4fa8-b5e4-2867e5a26742 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.792723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ff37-0594-4fa8-b5e4-2867e5a26742 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.793396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068ff37-0594-4fa8-b5e4-2867e5a26742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.794758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068ff37-0594-4fa8-b5e4-2867e5a26742 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.800325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 407/792] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:02.801758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b2028-379a-43a9-8096-727e69fba5eb None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.802049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b2028-379a-43a9-8096-727e69fba5eb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.802326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b2028-379a-43a9-8096-727e69fba5eb None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.803766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 408/793] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:02.805247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b2028-379a-43a9-8096-727e69fba5eb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:02.861864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.871494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde79eb-bfe8-4dce-b739-308a3450316f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:02.913054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205095c1-3ed8-4117-bb37-13e50b00dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:02.913784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205095c1-3ed8-4117-bb37-13e50b00dd57 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.914186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205095c1-3ed8-4117-bb37-13e50b00dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.914576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205095c1-3ed8-4117-bb37-13e50b00dd57 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.915566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205095c1-3ed8-4117-bb37-13e50b00dd57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:02.915946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205095c1-3ed8-4117-bb37-13e50b00dd57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:02.917514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205095c1-3ed8-4117-bb37-13e50b00dd57 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:02.919860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:02.946336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 386/794] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:02.948177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4769-b1bb-415a-952d-0b3b7bfe5ba3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:02.948307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4769-b1bb-415a-952d-0b3b7bfe5ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:02.948652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4769-b1bb-415a-952d-0b3b7bfe5ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:02.968007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5b6d4769-b1bb-415a-952d-0b3b7bfe5ba3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:02.969340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5b6d4769-b1bb-415a-952d-0b3b7bfe5ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:02.976064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b2028-379a-43a9-8096-727e69fba5eb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.022936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.023721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.023891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.024146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.025374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:03.025693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.027175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.041528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 409/795] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:03.042865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.059508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 387/796] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:03.060805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4769-b1bb-415a-952d-0b3b7bfe5ba3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.174642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:03.186698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93799131-9f27-4662-9590-19c94bb2ce3c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.197612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4769-b1bb-415a-952d-0b3b7bfe5ba3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.225367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:03.236979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.238776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.239199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.239549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.241263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.242341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.242801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.243251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.251064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.252242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.252993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.254451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:03.269380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:03.270905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:46:03.271833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:03.274537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 388/797] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:03.275559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:03.280882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:03.281770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:03.282657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:03.282657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:03.282657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:03.282657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:03.282657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 410/798] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:03.282928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.401084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b2b32-4f0d-4a69-bc82-a3c95bb052e0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.423277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fe9ce-1fe6-4be4-94ef-1f51d370eba0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.439011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.440008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.440562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.441140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.442211 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:03.442723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.444372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.462799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 411/799] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:03.464294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.466163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.467083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.467463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.468037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.469631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.471114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:03.487210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:03.488835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:03.489862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 389/800] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:03.490930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.496733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:03.543301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:03.596048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e94f-4d12-4609-903f-8150667b612c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.603182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7c68-48fc-44db-b21f-f2c7557e8502 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.639301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.640042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.640211 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.640632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.641002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.641492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.641910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:03.642053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.642402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.642627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.643973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.654450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.660835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 412/801] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:03.660835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.661072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:03.685920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:03.686867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:03.688381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:03.688381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:03.688381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:03.688381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:03.688381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 390/802] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:03.688381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.739015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:03.779590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea85a27-2ef7-49fe-a725-29996dda422e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.808771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:03.814737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.815503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.816440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.816714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8e3c-8438-4deb-8148-8951be9ebedc None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.816840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.817620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:03.817992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.819426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.824696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 413/803] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:03.825633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.858603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.859744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.860212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.860691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.868979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.870763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:03.888073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:03.889062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:03.889962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:03.897579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 391/804] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:03.898744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:03.931940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72380a6b-65d0-4743-81dc-50dc81cf6541 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:03.962599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:03.972530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:03.973759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:03.974047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:03.974329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:03.975572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:03.975884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:03.977340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:03.994042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 414/805] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:03.995364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:04.031113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020aab6-bb73-44b9-85e3-937490abf28e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:04.076563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:04.077808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.078769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.079252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.080426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:04.081108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:04.084038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:04.091100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 392/806] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:04.092530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:04.114631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f472b-2479-4821-a75e-08ce53169976 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:04.126027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:04.156024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:04.157015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.157015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.157272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.158043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:04.158351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:04.160342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:04.176484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 415/807] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:04.177463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:04.237099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5bf19-68dc-49f5-bfd5-b61b66cb25ef None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:04.239062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:04.287000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:04.288429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.288918 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.289618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.299181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe1176-35fa-4351-bfe4-c73bee5dc34d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:04.299754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:04.303273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:04.320801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:04.321720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:04.322695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:04.325624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 393/808] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:04.327088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:04.351716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:04.352415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.352689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.353024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.353768 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:04.355382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:04.380463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 416/809] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:04.381118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:04.435966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:04.459239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb856374-1dc9-495e-a8ce-a34d844240ae None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:04.506267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:04.511387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37926b0-df91-49b5-8ca9-b2b0bfc4ac51 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:04.512295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:04.513481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.514005 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.514502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/d95f13b6e27c43a7868a8ef79621eb7e/roles/94bbdda12f68411882dcceaf22124ca4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.532833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=d95f13b6e27c43a7868a8ef79621eb7e, role_id=94bbdda12f68411882dcceaf22124ca4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:04.534697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:04.562013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097247be-3899-4b56-8531-ebedb8fb1d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:04.562670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097247be-3899-4b56-8531-ebedb8fb1d33 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.563038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097247be-3899-4b56-8531-ebedb8fb1d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.563352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097247be-3899-4b56-8531-ebedb8fb1d33 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.564880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097247be-3899-4b56-8531-ebedb8fb1d33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:04.565309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097247be-3899-4b56-8531-ebedb8fb1d33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:04.568560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097247be-3899-4b56-8531-ebedb8fb1d33 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:04.575380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 394/810] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:46:04 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/d95f13b6e27c43a7868a8ef79621eb7e/roles/94bbdda12f68411882dcceaf22124ca4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:46:04.584610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 417/811] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:04.590011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:04.664930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d45b-1c25-413b-a7f5-4907ed614125 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.665265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d45b-1c25-413b-a7f5-4907ed614125 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.665475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d45b-1c25-413b-a7f5-4907ed614125 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.668471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 418/812] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:04.673586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96de185-f959-4191-878b-f97a7f36772f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.673898 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96de185-f959-4191-878b-f97a7f36772f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.674146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96de185-f959-4191-878b-f97a7f36772f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.699690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d96de185-f959-4191-878b-f97a7f36772f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:04.702009 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d96de185-f959-4191-878b-f97a7f36772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:04.759843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec635379-ac6e-464f-8e64-c31a80b2de27 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:04.796274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:04.807118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 419/813] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:04.807658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:04.808835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:04.809221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:04.809610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/31137a8197ab4072b47646b893a9733a/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:04.812203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=31137a8197ab4072b47646b893a9733a, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:04.814175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:04.834642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 395/814] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:46:04 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/31137a8197ab4072b47646b893a9733a/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:46:04.846321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96de185-f959-4191-878b-f97a7f36772f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:04.963828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:04.975426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96de185-f959-4191-878b-f97a7f36772f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:05.020341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3943f-2c74-44d3-b2fe-fd66202ac375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:05.021819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3943f-2c74-44d3-b2fe-fd66202ac375 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.022531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3943f-2c74-44d3-b2fe-fd66202ac375 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.023178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3943f-2c74-44d3-b2fe-fd66202ac375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.045484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3943f-2c74-44d3-b2fe-fd66202ac375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-10b3943f-2c74-44d3-b2fe-fd66202ac375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:05.054280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:05.055299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:05.056143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:05.058548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 420/815] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:05.064618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:05.198742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d484587-76b0-4d4c-bafe-8ef7988330b0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:05.219796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d63f-2b15-4ef3-a4e0-50a62b70f0cb None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.220341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d63f-2b15-4ef3-a4e0-50a62b70f0cb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.220341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d63f-2b15-4ef3-a4e0-50a62b70f0cb None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.222660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 421/816] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:05.228153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c36591-cea9-4faf-b618-88629f37533a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.228469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c36591-cea9-4faf-b618-88629f37533a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.228751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c36591-cea9-4faf-b618-88629f37533a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.239691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:05.240391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.240844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.241129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.241878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:05.242185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:05.243666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:05.246480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-91c36591-cea9-4faf-b618-88629f37533a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:05.247672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-91c36591-cea9-4faf-b618-88629f37533a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:05.252925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 396/817] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:46:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:05.261485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:05.354719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 422/818] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:05.367089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:05.377997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c36591-cea9-4faf-b618-88629f37533a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:05.437603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240db2ae-093f-4255-8225-6ee453c2a208 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:05.478502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:05.479659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.480174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.480675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.493603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:05.495538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:05.515581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:05.516546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:05.517436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:05.517436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:05.517436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:05.517436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:05.524784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 397/819] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:05.526414 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:05.562207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:05.623193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c36591-cea9-4faf-b618-88629f37533a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:05.648502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f160bac-daa5-467b-b43b-2cee3c92d6f0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:05.650638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:05.689377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:05.690430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.690989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.691571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.692540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:05.693076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:05.695301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:05.709636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 398/820] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:05.715275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:05.717190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:05.718062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.718348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.718623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] PATH_INFO: `/v3/services/network` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.719282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:05.720962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:05.741909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:05.743019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 15:46:05.743860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:05.753192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 423/821] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:05 2022] GET /identity/v3/services/network => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:05.768134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:05.852207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa6bf3-e762-4239-8b64-f66536007248 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:05.902914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:05.904079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:05.904596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:05.905105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:05.920612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:05.922514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:05.953414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1f4a-8b81-49a7-bdf3-e795ed66a84f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:05.956326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:05.957260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:05.958208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:05.968947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 399/822] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:46:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:05.971136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:06.024749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:06.033245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:06.034752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:06.035369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:06.036018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:06.037526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] RBAC: Adding query filter params (name=network) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:06.040320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:06.042723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:06.055889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 424/823] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:06.060978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:06.082210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:06.163721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f08d04-cf45-4e7a-af88-45c0b223fa5d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:06.237893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6226ed-5cd0-4f13-8af6-7a19d03d8f66 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:06.269742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:06.270439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:06.270722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:06.270995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:06.271844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:06.272148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:06.273630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:06.286357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:06.287048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:06.287331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:06.287602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:06.288317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] RBAC: Adding query filter params (type=network) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:06.288646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:06.293698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:06.307379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 425/824] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:06.307549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 400/825] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:46:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:06.313332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:06.313472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:06.441001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:06.455000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39f642-c973-4109-ac8c-2cb4326f7baa None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:06.493577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:06.494251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:06.494539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:06.495049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:06.495640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:06.496388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:06.497784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:06.525721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 401/826] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:06.550779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344aa47-b163-4e5b-9dad-632fb664f378 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:06.596146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:06.597447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:06.597867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:06.598150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:06.677633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:06.679114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:06.751231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:06.792380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 426/827] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:46:06.798074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:06.919402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28821488-9b27-4b04-85c2-96cec9890ae3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:06.962873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:06.963676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:06.963953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:06.964222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:06.967160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:06.977365 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:06.979463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:06.998913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:06.999847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:07.000791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:07.000791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:07.000791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:07.000791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:07.003198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 402/828] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:07.009107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.119921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21953795-f47b-4fc5-8088-0624474b9c0a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:07.158031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b063dc1-a9bf-4897-8546-5f67f92b9ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:07.158710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b063dc1-a9bf-4897-8546-5f67f92b9ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.158995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b063dc1-a9bf-4897-8546-5f67f92b9ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.159372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b063dc1-a9bf-4897-8546-5f67f92b9ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.160122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b063dc1-a9bf-4897-8546-5f67f92b9ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:07.160453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b063dc1-a9bf-4897-8546-5f67f92b9ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:07.161839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b063dc1-a9bf-4897-8546-5f67f92b9ff9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:07.174457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 427/829] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:07.180561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.297286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49cf9b-0add-4c92-a491-28fd3206b20a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:07.309345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66d172-989b-45f6-a5a8-02490924d129 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.309824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66d172-989b-45f6-a5a8-02490924d129 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.309899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66d172-989b-45f6-a5a8-02490924d129 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.311593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 428/830] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:07.317072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3b822-b3f7-4507-b76c-9d62fdd57cc4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.317363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3b822-b3f7-4507-b76c-9d62fdd57cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.317745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3b822-b3f7-4507-b76c-9d62fdd57cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.331881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:07.332641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.332920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.333191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.336800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-91a3b822-b3f7-4507-b76c-9d62fdd57cc4 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:07.337996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-91a3b822-b3f7-4507-b76c-9d62fdd57cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:07.340187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:07.341645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:07.357171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:07.358199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:07.359496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 403/831] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:07.365940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:07.366111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.417094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:07.432628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 429/832] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:07.450519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3b822-b3f7-4507-b76c-9d62fdd57cc4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.470462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38849e91-dd51-4cec-b415-1d22e317bfaf None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:07.509033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:07.509692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.509978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.510251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.511034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:07.511336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:07.512854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:07.530084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 404/833] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:07.531099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.589825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3b822-b3f7-4507-b76c-9d62fdd57cc4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:07.629594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaef60-d29a-4d9f-b7bc-3588b013a5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:07.630377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaef60-d29a-4d9f-b7bc-3588b013a5cc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.630656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaef60-d29a-4d9f-b7bc-3588b013a5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.630928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaef60-d29a-4d9f-b7bc-3588b013a5cc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.631568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daaef60-d29a-4d9f-b7bc-3588b013a5cc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:07.632946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daaef60-d29a-4d9f-b7bc-3588b013a5cc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1daaef60-d29a-4d9f-b7bc-3588b013a5cc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:07.644949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:07.645974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:07.646825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 430/834] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:07.647615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd15872-f6fc-4621-b3bc-01a73a9508c4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.647615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd15872-f6fc-4621-b3bc-01a73a9508c4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.647788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd15872-f6fc-4621-b3bc-01a73a9508c4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.649043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 431/835] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:07.651628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14f518-7f30-46d7-a5f0-8a8247e6a697 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.651628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14f518-7f30-46d7-a5f0-8a8247e6a697 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.651628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14f518-7f30-46d7-a5f0-8a8247e6a697 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.651844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 432/836] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:07.652450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14f518-7f30-46d7-a5f0-8a8247e6a697 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.654491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596f53f-51e9-4ff1-9c72-82ec3a06a88a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:07.665854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:07.675641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:07.692618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baba54-607e-47bd-b128-0391f6bba310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:07.693277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baba54-607e-47bd-b128-0391f6bba310 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.693556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baba54-607e-47bd-b128-0391f6bba310 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.693828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baba54-607e-47bd-b128-0391f6bba310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.694637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baba54-607e-47bd-b128-0391f6bba310 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0b352e6a51fc4d1f8b9b97d092d11bd7) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:07.694943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baba54-607e-47bd-b128-0391f6bba310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:07.696380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baba54-607e-47bd-b128-0391f6bba310 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:07.702097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 405/837] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/role_assignments?user.id=0b352e6a51fc4d1f8b9b97d092d11bd7&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:07.703200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f447a-7a59-4a57-9aa9-31705ec59823 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.703478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f447a-7a59-4a57-9aa9-31705ec59823 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.703750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f447a-7a59-4a57-9aa9-31705ec59823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.726841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d34f447a-7a59-4a57-9aa9-31705ec59823 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:07.728267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d34f447a-7a59-4a57-9aa9-31705ec59823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:07.767125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14f518-7f30-46d7-a5f0-8a8247e6a697 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:07.810336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:07.810599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 406/838] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:07.811523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.811711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a65e67-3c2c-45dc-b5f3-e4cbaa50619d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.812011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a65e67-3c2c-45dc-b5f3-e4cbaa50619d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.812225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a65e67-3c2c-45dc-b5f3-e4cbaa50619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.812707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.813191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.814133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:07.814633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:07.816579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:07.825271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 433/839] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:07.830484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.842197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-99a65e67-3c2c-45dc-b5f3-e4cbaa50619d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:07.843215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-99a65e67-3c2c-45dc-b5f3-e4cbaa50619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:07.919294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 407/840] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:07.920061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a65e67-3c2c-45dc-b5f3-e4cbaa50619d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:07.938664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:07.958639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb9a64-d9fc-4721-9510-bc6cf7bac32c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:07.959929 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:07.997567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:07.998236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:07.998519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:07.998792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:07.999497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:07.999801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.001200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.006683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 434/841] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:08.007771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.045273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a65e67-3c2c-45dc-b5f3-e4cbaa50619d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.086924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.087844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.088127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.088443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.102187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:08.110591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:08.111597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:08.112596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 408/842] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:46:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:08.118586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.135672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b227-6cd5-4c8f-88d8-99967ff2d242 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.191243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.192750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.193512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.193995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.210395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:08.219049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:08.220054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:08.220924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:08.222355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 435/843] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:08.228639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.240319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:08.249587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:08.261163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3d207-e9a6-48fb-b788-0febc6aea1be None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.306332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.307012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.307297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.307577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.308618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:08.308935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.311301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.327704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 409/844] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:08.330377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.352589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625e19d-dd77-42db-a453-51c7a0e04613 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.393005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.394354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.394642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.395015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.395891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:08.396271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.398007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.407606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 436/845] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:08.413787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.464401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe837327-bba7-4714-8b57-bafec31b0a96 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.506891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:08.513734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.514666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.515053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.515434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.531303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.533252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.544516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7fe58-1d62-41e4-b559-7f0e2ad686fe None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:08.557070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:08.558099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:08.558969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:08.558969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:08.558969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:08.558969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:08.561178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 410/846] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:08.566549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.578723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:08.593169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.593973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.594248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.594521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.606638 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.608233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:08.626982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:08.627978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:08.628945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:08.628945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:08.628945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:08.628945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:08.628945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 437/847] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:08.634012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.708320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a6fbc-8b31-41c7-be61-7ca573fd2c9b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.739186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428e3b-4e4b-483d-9c82-0b9a85400fbb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.744913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.745582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.745863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.746137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.746852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:08.747154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.748642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.765866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 411/848] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:08.766856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.770140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:08.786111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.786844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.787128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.787401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.788562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:08.788872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.790770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.807461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 438/849] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:08.812238 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.851893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:08.890562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522f23b-e886-4a0d-ba33-018e03049027 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.922243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2fe96-da48-4256-a285-954348312182 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:08.929644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.930468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.930754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.931037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.939153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.941230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:08.957659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:08.958646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:08.959576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:08.963665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 412/850] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:08.964706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:08.966807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:08.967606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:08.967885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:08.968155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:08.983679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:08.986612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:09.004284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:09.005373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 15:46:09.006307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:09.009059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 439/851] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:09.017420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.064630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.073705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023470-a3f4-48a9-a8a4-764f15c85607 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.104182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.114852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.115636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.115925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.116207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.117122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:09.117383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.118843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.141724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf59192-4846-42e7-91e0-0ecac69ee3e2 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.145607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 413/852] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Jun 7 15:46:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:09.146651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.182974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.183777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.184163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.184570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.185510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:09.185920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.187434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.208324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 440/853] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:09.213897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.281778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fcc02-e4a8-4e49-8c1f-634a5eef428b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.327478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.328393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.328685 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.329368 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.336816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.342460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf474e-2bf0-456c-8675-edb1a0a92483 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.381168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.381971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.382617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.382900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.388577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.406265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.407664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.471044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.472929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.519720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 414/854] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:46:09.520668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.579272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.588720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 441/855] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:46:09.590040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.610774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.630478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae86ef-3e3d-4a9e-9bbf-5f44a1fb27be None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.669129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.669846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.670234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.670520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.682711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.684185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.693933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e1d30-146f-4274-bd75-a06364692531 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:09.703570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:09.705614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:09.707794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:09.707794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:09.707794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:09.707794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:09.709745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 415/856] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:09.716377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.729526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.730318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.730594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.730862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.742240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.743719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:09.763495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:09.765752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:09.767910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:09.767910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:09.767910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:09.767910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:09.776439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 442/857] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:09.776439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.826307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.837953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2cd62-c3a3-4e0b-904f-2c35bc2a1f6c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.877782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1721-c2fa-4b3e-a169-a644770700b1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:09.885643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.886317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.886593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.886889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.888096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:09.888432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.890255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.894092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:09.915024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 416/858] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:09.921696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:09.925877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:09.926735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:09.927130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:09.927510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:09.928342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:09.928774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:09.930242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:09.947508 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 443/859] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:09.948692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.047230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27904ea-faa8-4b9f-8d83-62743d6484fb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.069942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2680e7-979f-446a-906c-7b13fad744eb None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.085667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1e83b-578d-4024-b672-301b8b4e96a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.086089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:10.086924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1e83b-578d-4024-b672-301b8b4e96a1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.087370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1e83b-578d-4024-b672-301b8b4e96a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.087803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1e83b-578d-4024-b672-301b8b4e96a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.095310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1e83b-578d-4024-b672-301b8b4e96a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.097391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1e83b-578d-4024-b672-301b8b4e96a1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.111088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1269d-c587-4826-9423-8798f85c85d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.111888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1269d-c587-4826-9423-8798f85c85d8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.112175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1269d-c587-4826-9423-8798f85c85d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-73b1e83b-578d-4024-b672-301b8b4e96a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:10.112309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:10.113345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:10.114396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:10.114851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1269d-c587-4826-9423-8798f85c85d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.114935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 417/860] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:10.114935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73af7a9-b31d-49a2-8f08-3f6073b003e8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.115565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73af7a9-b31d-49a2-8f08-3f6073b003e8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.115565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73af7a9-b31d-49a2-8f08-3f6073b003e8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.116825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 418/861] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:10.121203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73af7a9-b31d-49a2-8f08-3f6073b003e8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.125957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1269d-c587-4826-9423-8798f85c85d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.127406 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1269d-c587-4826-9423-8798f85c85d8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-f1e1269d-c587-4826-9423-8798f85c85d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:10.147282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:10.148208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:10.149131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 444/862] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:10.150646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68639d-063a-4b47-96b2-31cc8417404e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.150994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68639d-063a-4b47-96b2-31cc8417404e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.151339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68639d-063a-4b47-96b2-31cc8417404e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.170157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bc68639d-063a-4b47-96b2-31cc8417404e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:10.171342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bc68639d-063a-4b47-96b2-31cc8417404e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:10.195711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:10.246850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73af7a9-b31d-49a2-8f08-3f6073b003e8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.249902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 445/863] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:10.250739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68639d-063a-4b47-96b2-31cc8417404e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.294773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.295845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.296403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.296597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.297379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:10.297698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.299129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.313764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 419/864] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:10.314902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.355872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68639d-063a-4b47-96b2-31cc8417404e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.358124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:10.395443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.396095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.396480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.396755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.397538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:10.397843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.399274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.411579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 446/865] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:10.412398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.430050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aface01-c3ae-4d2e-97df-61fd3634eeb3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.446106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:10.464193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7cc7a-7e68-4d70-bc73-2967b6066d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.465028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7cc7a-7e68-4d70-bc73-2967b6066d5e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.465306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7cc7a-7e68-4d70-bc73-2967b6066d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.465577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7cc7a-7e68-4d70-bc73-2967b6066d5e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.466216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7cc7a-7e68-4d70-bc73-2967b6066d5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.467576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7cc7a-7e68-4d70-bc73-2967b6066d5e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5fb7cc7a-7e68-4d70-bc73-2967b6066d5e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:10.480532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:10.481572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:10.482405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 420/866] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Jun 7 15:46:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:10.483280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53e3c4-3301-49f7-934c-2bbbf2e38049 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.483280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53e3c4-3301-49f7-934c-2bbbf2e38049 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.483413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53e3c4-3301-49f7-934c-2bbbf2e38049 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.484806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 421/867] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:10.485327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53e3c4-3301-49f7-934c-2bbbf2e38049 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.518614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275e781-9584-42ee-b656-63b0c211f399 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.552800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.553456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.553742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.554013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.554833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=d95f13b6e27c43a7868a8ef79621eb7e) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:10.555142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.556577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.562322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 447/868] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:10 2022] GET /identity/v3/role_assignments?user.id=d95f13b6e27c43a7868a8ef79621eb7e&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:10.563123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.594113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:10.596081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53e3c4-3301-49f7-934c-2bbbf2e38049 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.636757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c940bdb-d3d8-4b25-aa2a-e303c06e947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.637839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c940bdb-d3d8-4b25-aa2a-e303c06e947b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.638199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c940bdb-d3d8-4b25-aa2a-e303c06e947b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.638537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c940bdb-d3d8-4b25-aa2a-e303c06e947b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.639435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c940bdb-d3d8-4b25-aa2a-e303c06e947b None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=31137a8197ab4072b47646b893a9733a) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:10.639805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c940bdb-d3d8-4b25-aa2a-e303c06e947b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.641306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c940bdb-d3d8-4b25-aa2a-e303c06e947b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.660817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 422/869] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:10 2022] GET /identity/v3/role_assignments?user.id=31137a8197ab4072b47646b893a9733a&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:10.663640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928acccd-a666-42be-be83-edb789edb1a7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.664267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928acccd-a666-42be-be83-edb789edb1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.665096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928acccd-a666-42be-be83-edb789edb1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.690019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de8578-3029-42d1-993b-28d5c1f0ae81 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.691752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-928acccd-a666-42be-be83-edb789edb1a7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:10.694720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-928acccd-a666-42be-be83-edb789edb1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:10.732858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.733459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.733738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.734016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.734739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:10.735048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.736618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.747548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 448/870] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:10.750001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.762008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:10.797862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 423/871] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:10.822128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928acccd-a666-42be-be83-edb789edb1a7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:10.874641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:10.886215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55188c47-e82a-4a7e-8a5c-6d031cec75b4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.928606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab67ab-932b-4a65-bd07-afddd9e6adcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.929271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab67ab-932b-4a65-bd07-afddd9e6adcb None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.929548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab67ab-932b-4a65-bd07-afddd9e6adcb None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.929817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab67ab-932b-4a65-bd07-afddd9e6adcb None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.930515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab67ab-932b-4a65-bd07-afddd9e6adcb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:10.930814 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab67ab-932b-4a65-bd07-afddd9e6adcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:10.932163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab67ab-932b-4a65-bd07-afddd9e6adcb None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:10.937435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 449/872] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:10.952489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928acccd-a666-42be-be83-edb789edb1a7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:10.997130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c981ecf-d7be-4793-abf3-25149322d4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:10.997729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c981ecf-d7be-4793-abf3-25149322d4b9 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:10.998016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c981ecf-d7be-4793-abf3-25149322d4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:10.998297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c981ecf-d7be-4793-abf3-25149322d4b9 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:10.999039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c981ecf-d7be-4793-abf3-25149322d4b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:11.000465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c981ecf-d7be-4793-abf3-25149322d4b9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:11.018496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 424/873] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:46:10 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:12.956895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5254aef-7db6-4224-b81a-e31c6f76414a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:12.958014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5254aef-7db6-4224-b81a-e31c6f76414a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:12.958506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5254aef-7db6-4224-b81a-e31c6f76414a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:12.960343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 450/874] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:12.966079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9366090-6580-4f36-b1d6-25cac9739d66 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:12.966491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9366090-6580-4f36-b1d6-25cac9739d66 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:12.966697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9366090-6580-4f36-b1d6-25cac9739d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:12.984591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b9366090-6580-4f36-b1d6-25cac9739d66 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:12.985769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b9366090-6580-4f36-b1d6-25cac9739d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:13.016070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:13.028586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7a507-3f91-4965-9e37-2ab2e6c852b0 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.028880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7a507-3f91-4965-9e37-2ab2e6c852b0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.029148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7a507-3f91-4965-9e37-2ab2e6c852b0 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.030801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 451/875] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:13.036701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3380cb-6678-4d40-a2fc-9c5c2877e64f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.037055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3380cb-6678-4d40-a2fc-9c5c2877e64f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.037315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3380cb-6678-4d40-a2fc-9c5c2877e64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.055560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1c3380cb-6678-4d40-a2fc-9c5c2877e64f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:13.056810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1c3380cb-6678-4d40-a2fc-9c5c2877e64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:13.080227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 425/876] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:12 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:13.100252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9366090-6580-4f36-b1d6-25cac9739d66 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.141913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 452/877] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:13.169915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3380cb-6678-4d40-a2fc-9c5c2877e64f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.206707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9366090-6580-4f36-b1d6-25cac9739d66 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.247532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537b13d-a4e5-4ad5-978b-f21cc16fb54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.248612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b13d-a4e5-4ad5-978b-f21cc16fb54b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.249071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b13d-a4e5-4ad5-978b-f21cc16fb54b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.249502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b13d-a4e5-4ad5-978b-f21cc16fb54b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.262572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537b13d-a4e5-4ad5-978b-f21cc16fb54b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:13.262758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6537b13d-a4e5-4ad5-978b-f21cc16fb54b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:13.271024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:13.271993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:13.272887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 426/878] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:13.274174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62723-7ec2-4474-baee-b6f541f3363b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.274174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62723-7ec2-4474-baee-b6f541f3363b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.274174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62723-7ec2-4474-baee-b6f541f3363b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.275419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 427/879] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:13.278111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f62723-7ec2-4474-baee-b6f541f3363b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.285999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3380cb-6678-4d40-a2fc-9c5c2877e64f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.302450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:13.321609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42312328-add0-4310-888d-172d81cb6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.322406 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42312328-add0-4310-888d-172d81cb6c8f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.322678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42312328-add0-4310-888d-172d81cb6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.322948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42312328-add0-4310-888d-172d81cb6c8f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.323584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42312328-add0-4310-888d-172d81cb6c8f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:13.324971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42312328-add0-4310-888d-172d81cb6c8f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-42312328-add0-4310-888d-172d81cb6c8f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:13.337212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:13.338239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:13.339084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 453/880] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:13.339875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6996-0780-4611-a6e0-89c5e30354b0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.339875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6996-0780-4611-a6e0-89c5e30354b0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.340554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6996-0780-4611-a6e0-89c5e30354b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.361797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0cdd6996-0780-4611-a6e0-89c5e30354b0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:13.363692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0cdd6996-0780-4611-a6e0-89c5e30354b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:13.393487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f62723-7ec2-4474-baee-b6f541f3363b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.427257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d5198-ceb4-47ec-a5d0-6aa636eef6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.427922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5198-ceb4-47ec-a5d0-6aa636eef6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.428213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5198-ceb4-47ec-a5d0-6aa636eef6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.428521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5198-ceb4-47ec-a5d0-6aa636eef6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.429259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d5198-ceb4-47ec-a5d0-6aa636eef6e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:13.429568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d5198-ceb4-47ec-a5d0-6aa636eef6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:13.430932 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d5198-ceb4-47ec-a5d0-6aa636eef6e4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:13.439954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 454/881] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:13.439954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd6996-0780-4611-a6e0-89c5e30354b0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.440176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 428/882] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:13.444004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952072cd-fb24-48d0-be60-4b4c44001b88 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.444004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952072cd-fb24-48d0-be60-4b4c44001b88 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.444004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952072cd-fb24-48d0-be60-4b4c44001b88 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.444805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 429/883] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:13.445894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952072cd-fb24-48d0-be60-4b4c44001b88 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.511385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:13.531175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:13.533592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd6996-0780-4611-a6e0-89c5e30354b0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.551052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952072cd-fb24-48d0-be60-4b4c44001b88 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.566427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95152b22-2351-450a-81a3-cb6a07a820c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.567077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95152b22-2351-450a-81a3-cb6a07a820c0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.567354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95152b22-2351-450a-81a3-cb6a07a820c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.567619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95152b22-2351-450a-81a3-cb6a07a820c0 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.568313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95152b22-2351-450a-81a3-cb6a07a820c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:13.568639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95152b22-2351-450a-81a3-cb6a07a820c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:13.569983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95152b22-2351-450a-81a3-cb6a07a820c0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:13.574659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 455/884] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:13.575870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8865a2-f795-4f85-8fd7-7c702f23d89d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.576276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8865a2-f795-4f85-8fd7-7c702f23d89d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.576579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8865a2-f795-4f85-8fd7-7c702f23d89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.589501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.590282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.590561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.590829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.592459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3d8865a2-f795-4f85-8fd7-7c702f23d89d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:13.593631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3d8865a2-f795-4f85-8fd7-7c702f23d89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:13.602874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:13.604296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:13.621449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:13.622515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:13.623437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:13.623437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:13.623437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:13.623437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:13.623437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 430/885] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:13.623437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.661509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 456/886] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:13.662341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8865a2-f795-4f85-8fd7-7c702f23d89d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.724241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5423a3e-36bb-4cd2-91d5-b8f8ea9f5428 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.743663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:13.757888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.758544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.758822 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.759092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.759800 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:13.761183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:13.763274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:13.775420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8865a2-f795-4f85-8fd7-7c702f23d89d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.776111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 431/887] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:46:13 2022] POST /identity/v3/services => generated 269 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:13.776852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.825737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.826414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.826699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.826973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.827695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:13.827998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:13.829410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:13.836037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 457/888] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:13.837044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:13.932524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbfc57-4d4c-4a53-90b5-31e2ca2aeb1a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:13.984721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:13.985402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:13.985909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:13.986199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:13.988130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:13.988475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:13.989900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:13.990086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdb976-4639-4324-8492-5de53ce5a0c8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.007876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 432/889] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:14.014086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.032965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:14.044165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.045042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.045327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.045600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.046285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.048211 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:14.064104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:14.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:14.066569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 458/890] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:14.072584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.092602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:14.161716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d06e87-2b91-499b-87d6-c5054e47db55 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.200606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.201565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.201943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.202284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.202651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50900f4-f4f2-49b0-8f13-edf2b4fec4ad None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.212118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.214285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:14.231712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:14.232940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:14.234104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:14.234779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 433/891] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:14.240210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.254040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.255253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.255695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.256157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.257735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:14.258218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.260270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.272016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 459/892] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:14.273551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.354539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:14.359951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ba18d-8f45-450f-a603-5d28d9edab73 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.377890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:14.402994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.403772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.404171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.404546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.405606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:14.405980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.407588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.417237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23036e-dae8-4716-8b83-48f2e25177e1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.432310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 434/893] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:14.433322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.462004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.463220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.464109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.464862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.466000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:14.466496 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.468725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.480701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 460/894] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:14.481524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.565028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b07a8-d820-4753-83d1-9d363557c768 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.598517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829634a-bac5-46a7-b70d-0ac2f4aa5a7c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.611115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.613064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.614067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.614676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.624344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:14.638339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.639141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.639487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.639837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.640932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] RBAC: Adding query filter params (interface=public, service_id=92df4d0e32f1422cb449eff4b6c0d263, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:14.641651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.643094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.658415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 461/895] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/endpoints?service_id=92df4d0e32f1422cb449eff4b6c0d263&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:14.660250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.671649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:14.703376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.704877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.794863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f72e88-fd24-4a39-9193-ddb99b190a4b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.817009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 435/896] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:46:14.818222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:14.838143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.839054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.839530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.839995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.841075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.842526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:14.856381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 462/897] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:46:14.904435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:14.937797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bee01f-0485-4bc4-85b0-e5cd08fdb949 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:14.976823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:14.977661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:14.977946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:14.978226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:14.990932 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:14.992500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:15.012235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:15.013279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:15.014154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:15.014154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:15.014154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:15.014154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:15.014154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 436/898] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:15.019538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:15.088470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:15.140650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675b870-d3ff-48a0-acfc-8f450e262e0e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:15.176179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:15.176874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:15.177158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:15.177432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:15.178150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:15.178453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:15.179828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:15.192841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 463/899] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:15.198236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:15.301961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dad48f-adf4-4223-b70f-b0bb8ce1526b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:15.327449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:15.342775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:15.343675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:15.343960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:15.344231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:15.352027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:15.353492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:15.367940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:15.368839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:15.369736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 437/900] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:15.376245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:15.481319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa3ee8-9cbf-4dda-ba35-f2167ec296c5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:15.496720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:15.519472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d675-8468-4ae4-a8c4-3d9354ab954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:15.520535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d675-8468-4ae4-a8c4-3d9354ab954e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:15.521014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d675-8468-4ae4-a8c4-3d9354ab954e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:15.521491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d675-8468-4ae4-a8c4-3d9354ab954e None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:15.522460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870d675-8468-4ae4-a8c4-3d9354ab954e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:15.522958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870d675-8468-4ae4-a8c4-3d9354ab954e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:15.524604 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870d675-8468-4ae4-a8c4-3d9354ab954e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:15.537830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 464/901] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:15.545419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:15.648989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2318ee-1814-47c2-a51b-bb43ff1987d5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:15.683945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fd08d-91a9-4028-84f0-7c40d16d8441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:15.684634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fd08d-91a9-4028-84f0-7c40d16d8441 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:15.684924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fd08d-91a9-4028-84f0-7c40d16d8441 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:15.685202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fd08d-91a9-4028-84f0-7c40d16d8441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:15.686022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fd08d-91a9-4028-84f0-7c40d16d8441 None admin] RBAC: Adding query filter params (role.id=125bd519889240d29b02cfd21e83bab8, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=d95f13b6e27c43a7868a8ef79621eb7e) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:15.686333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fd08d-91a9-4028-84f0-7c40d16d8441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:15.687768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fd08d-91a9-4028-84f0-7c40d16d8441 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:15.694717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 438/902] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:15 2022] GET /identity/v3/role_assignments?user.id=d95f13b6e27c43a7868a8ef79621eb7e&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=125bd519889240d29b02cfd21e83bab8 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:16.227294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775cdf-0051-4f7b-bc0e-3e0f270c0a2a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.227808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775cdf-0051-4f7b-bc0e-3e0f270c0a2a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.227928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775cdf-0051-4f7b-bc0e-3e0f270c0a2a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.229522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 465/903] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:16.234927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df6f7-6d54-450a-a7f6-ad9772b928fb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.235353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df6f7-6d54-450a-a7f6-ad9772b928fb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.235530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df6f7-6d54-450a-a7f6-ad9772b928fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.253664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a61df6f7-6d54-450a-a7f6-ad9772b928fb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:16.254911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a61df6f7-6d54-450a-a7f6-ad9772b928fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:16.263875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff04536-8e20-483b-8b7e-70e60e9d92d3 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.264462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff04536-8e20-483b-8b7e-70e60e9d92d3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.264959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff04536-8e20-483b-8b7e-70e60e9d92d3 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.270108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 466/904] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:16.271886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7bf7c-bf1b-4090-b9cf-e6977334dbb6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.272176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7bf7c-bf1b-4090-b9cf-e6977334dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.272527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7bf7c-bf1b-4090-b9cf-e6977334dbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.291347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cdc7bf7c-bf1b-4090-b9cf-e6977334dbb6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:16.292558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cdc7bf7c-bf1b-4090-b9cf-e6977334dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:16.316436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:16.356402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 439/905] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:16.371069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 467/906] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:16.377253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df6f7-6d54-450a-a7f6-ad9772b928fb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:16.397355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7bf7c-bf1b-4090-b9cf-e6977334dbb6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:16.490904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df6f7-6d54-450a-a7f6-ad9772b928fb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:16.521612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7bf7c-bf1b-4090-b9cf-e6977334dbb6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:16.523632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:16.534569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:16.535402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.535704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.535948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.537024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:16.538748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:16.555878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:16.556931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:16.557797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 440/907] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Jun 7 15:46:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:16.563408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:16.570173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:16.571190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.571673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.572143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.573119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:16.574749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:16.592553 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 468/908] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:46:16 2022] POST /identity/v3/services => generated 241 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:46:16.624698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:16.693319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827929e-58ad-4389-b6ba-8bbb383be5d3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:16.740804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:16.741933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.742435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.742909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.743835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:16.744373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:16.746049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:16.752258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 441/909] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:16.761575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:16.887846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6c1e-8c30-48d0-9ba6-997051d6390c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:16.928643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:16.929354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:16.929644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:16.929924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:16.930665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:16.930974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:16.932444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:16.939609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 469/910] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:16.945812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:17.078956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af3144-3295-44fa-8fa2-56b5e31ef6ec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:17.081553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:17.130057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af225996-2470-4c85-90c6-775adebc3e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:17.130946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af225996-2470-4c85-90c6-775adebc3e5c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:17.131143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af225996-2470-4c85-90c6-775adebc3e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:17.131422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af225996-2470-4c85-90c6-775adebc3e5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:17.132214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af225996-2470-4c85-90c6-775adebc3e5c None admin] RBAC: Adding query filter params (interface=public, service_id=6b3151be5cbc4ab797732cdf20c1d62b, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:17.132574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af225996-2470-4c85-90c6-775adebc3e5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:17.133958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af225996-2470-4c85-90c6-775adebc3e5c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:17.140203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 442/911] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:46:16 2022] GET /identity/v3/endpoints?service_id=6b3151be5cbc4ab797732cdf20c1d62b&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:17.146405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:17.181918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:17.270102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d115f6b-774a-4217-a485-4885c05c30b6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:17.310785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1f9af-831e-4699-8bb8-bcd14521c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:17.311851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f9af-831e-4699-8bb8-bcd14521c647 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:17.312330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f9af-831e-4699-8bb8-bcd14521c647 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:17.312842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f9af-831e-4699-8bb8-bcd14521c647 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:17.313716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1f9af-831e-4699-8bb8-bcd14521c647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:17.315817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1f9af-831e-4699-8bb8-bcd14521c647 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:17.326864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5280e-8121-4fb7-8bf3-9e817e1a2585 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:17.327399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5280e-8121-4fb7-8bf3-9e817e1a2585 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:17.327688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 470/912] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:46:17 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:46:17.327788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5280e-8121-4fb7-8bf3-9e817e1a2585 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:17.336151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 443/913] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:17.338036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb1873-44a2-46a6-a1ec-748caf100913 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:17.338324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb1873-44a2-46a6-a1ec-748caf100913 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:17.338598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb1873-44a2-46a6-a1ec-748caf100913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:17.358724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f5fb1873-44a2-46a6-a1ec-748caf100913 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:17.359965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f5fb1873-44a2-46a6-a1ec-748caf100913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:17.447089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 471/914] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:17.470074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5280e-8121-4fb7-8bf3-9e817e1a2585 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:17.590361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5280e-8121-4fb7-8bf3-9e817e1a2585 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:17.628914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:17.630057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:17.630557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:17.630969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:17.632078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:17.633678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:17.647116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:17.648462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 15:46:17.649500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:17.652159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 444/915] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:17.657822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb1873-44a2-46a6-a1ec-748caf100913 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:17.680026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:17.760448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb1873-44a2-46a6-a1ec-748caf100913 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:17.800323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:17.801026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:17.801314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:17.801585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:17.802297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:17.802599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:17.803998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:17.809699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 472/916] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:17.815740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:17.849189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:17.927380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a14f7-d22d-452f-ad0a-46081c19ce7a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:17.965794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5551a6-64e1-4c51-8588-c703cd948c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:17.966469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5551a6-64e1-4c51-8588-c703cd948c3e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:17.966752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5551a6-64e1-4c51-8588-c703cd948c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:17.967033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5551a6-64e1-4c51-8588-c703cd948c3e None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:17.967737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5551a6-64e1-4c51-8588-c703cd948c3e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:17.968044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5551a6-64e1-4c51-8588-c703cd948c3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:17.969436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5551a6-64e1-4c51-8588-c703cd948c3e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:17.975496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 445/917] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Jun 7 15:46:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:17.981206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:18.072753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:18.088178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98847e59-3b9c-4da7-abaf-545f3cfa095d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:18.124549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17df87-ea66-4d3a-9db2-1ec1e6b97687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:18.125135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17df87-ea66-4d3a-9db2-1ec1e6b97687 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:18.125416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17df87-ea66-4d3a-9db2-1ec1e6b97687 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:18.125692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17df87-ea66-4d3a-9db2-1ec1e6b97687 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:18.126442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17df87-ea66-4d3a-9db2-1ec1e6b97687 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:18.127838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17df87-ea66-4d3a-9db2-1ec1e6b97687 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:18.150869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def053b9-acc8-4d92-a6a7-7a653b239f0b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:18.151424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def053b9-acc8-4d92-a6a7-7a653b239f0b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:18.151956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def053b9-acc8-4d92-a6a7-7a653b239f0b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:18.153305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 473/918] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:18.153828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 446/919] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:18.160243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b82cf-0b1d-4a25-8e57-40735f2e1161 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:18.160591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b82cf-0b1d-4a25-8e57-40735f2e1161 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:18.160830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b82cf-0b1d-4a25-8e57-40735f2e1161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:18.187516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f35b82cf-0b1d-4a25-8e57-40735f2e1161 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:18.188840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f35b82cf-0b1d-4a25-8e57-40735f2e1161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:18.281502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 474/920] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:18.306140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def053b9-acc8-4d92-a6a7-7a653b239f0b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:18.416390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:18.425025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def053b9-acc8-4d92-a6a7-7a653b239f0b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:18.462079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:18.462938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:18.463215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:18.463538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:18.475578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:18.483712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:18.484690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:18.485518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 447/921] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Jun 7 15:46:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:18.492896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b82cf-0b1d-4a25-8e57-40735f2e1161 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:18.564439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:18.600993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b82cf-0b1d-4a25-8e57-40735f2e1161 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:18.641537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:18.642209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:18.642491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:18.642764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:18.643564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:18.643867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:18.645262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:18.654155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 475/922] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Jun 7 15:46:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:18.660375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:18.765775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828ee27-3771-4dbe-8f9b-a6b7b846dc2a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:18.805835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:18.806636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:18.806917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:18.807191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:18.818418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:18.819857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:18.838082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:18.839124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:18.840009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:18.840009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:18.840009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:18.840009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:18.840009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 448/923] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:18.846272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:18.952229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddf73d-d58d-4be2-84cd-97e4ef6ae794 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:18.979105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:18.987637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:18.988350 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:18.988660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:18.988939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:18.989661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:18.989967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:18.991432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.004093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 476/924] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:19.009554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.023246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:19.123958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170d8b9-9d53-4d78-9e5c-dc88e57cd528 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.162200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c079a-c57d-4a40-975e-55cbe6606866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.163023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c079a-c57d-4a40-975e-55cbe6606866 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.163310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c079a-c57d-4a40-975e-55cbe6606866 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.163586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c079a-c57d-4a40-975e-55cbe6606866 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.171753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c079a-c57d-4a40-975e-55cbe6606866 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.173222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c079a-c57d-4a40-975e-55cbe6606866 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f22c079a-c57d-4a40-975e-55cbe6606866 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:19.188151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:19.189152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:19.190130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 449/925] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:19.195344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.228475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be6ca-1463-49c2-87db-4ddda17969f3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.228780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be6ca-1463-49c2-87db-4ddda17969f3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.229026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be6ca-1463-49c2-87db-4ddda17969f3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.230970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 450/926] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:19.235970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49f3ad-2382-48ff-b558-ec3ea3ef90d5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.236255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49f3ad-2382-48ff-b558-ec3ea3ef90d5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.236558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49f3ad-2382-48ff-b558-ec3ea3ef90d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.255484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2b49f3ad-2382-48ff-b558-ec3ea3ef90d5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:19.256744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2b49f3ad-2382-48ff-b558-ec3ea3ef90d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:19.307971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c6ecb-1054-49de-88bf-a7d4d3ef4ff1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.321056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:19.335557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 451/927] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:19.347167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.348321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.348888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.349383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.350689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:19.351206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.352896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.355749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49f3ad-2382-48ff-b558-ec3ea3ef90d5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.373445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 477/928] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:19.382007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.440844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:19.477822 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49f3ad-2382-48ff-b558-ec3ea3ef90d5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.518908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.519751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.520040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.520305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.520559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56b50-2c55-4c21-a6d0-ee0613f8f4b7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.520989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.522372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:19.535397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:19.536480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:19.537310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:19.540617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 452/929] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:19.542842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.561601 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.562321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.562611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.562866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.574249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.575719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:19.593997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:19.597412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:19.600696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:19.600696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:19.600696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:19.600696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:19.600696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 478/930] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:19.612646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.618150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:19.674991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f36955-cbae-4d27-8143-f1fbfd76dc54 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.713578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:19.716292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.716995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.717349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.717640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.718405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:19.718648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76a5f-2b5e-43f6-b350-429181900fe0 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.718782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.720062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.731566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 453/931] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:19.732814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.762837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.763987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.764839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.765410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.766408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:19.767108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.769850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.789188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 479/932] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:19.793498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.856678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100155e-8bf3-43a6-ab1d-96f69a28026b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.885152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:19.899003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb17e9-93f7-4018-9bb9-2a72e3a33f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.899960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb17e9-93f7-4018-9bb9-2a72e3a33f46 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.900461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb17e9-93f7-4018-9bb9-2a72e3a33f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.900917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb17e9-93f7-4018-9bb9-2a72e3a33f46 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.901831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb17e9-93f7-4018-9bb9-2a72e3a33f46 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:19.902289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb17e9-93f7-4018-9bb9-2a72e3a33f46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.903848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb17e9-93f7-4018-9bb9-2a72e3a33f46 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.909464 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 454/933] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:19.910965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f721a-1198-4774-ab7e-edfb9fbf702e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.911253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f721a-1198-4774-ab7e-edfb9fbf702e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.911530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f721a-1198-4774-ab7e-edfb9fbf702e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.912952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 455/934] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:19.915083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f721a-1198-4774-ab7e-edfb9fbf702e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:19.925146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b87b0e-fd35-461b-a274-42fa8f850281 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:19.964688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06022-0cf2-44bc-b3c0-48c4ee1cd3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:19.965494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06022-0cf2-44bc-b3c0-48c4ee1cd3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.965764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06022-0cf2-44bc-b3c0-48c4ee1cd3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.966044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06022-0cf2-44bc-b3c0-48c4ee1cd3d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:19.973454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a06022-0cf2-44bc-b3c0-48c4ee1cd3d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:19.975153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a06022-0cf2-44bc-b3c0-48c4ee1cd3d9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-25a06022-0cf2-44bc-b3c0-48c4ee1cd3d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:19.990184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:19.991119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:19.992021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 480/935] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:19.993305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704317e-a11f-4848-9907-8ee9ec951931 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:19.993305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704317e-a11f-4848-9907-8ee9ec951931 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:19.993549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704317e-a11f-4848-9907-8ee9ec951931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.010863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c704317e-a11f-4848-9907-8ee9ec951931 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:20.012054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c704317e-a11f-4848-9907-8ee9ec951931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:20.025907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:20.045288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f721a-1198-4774-ab7e-edfb9fbf702e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.081780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.082442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.082727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.083008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.083773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] RBAC: Adding query filter params (interface=public, service_id=b7c8d3120b304c8487d18462e239504d, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:20.084086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.085699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.087150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 481/936] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:20.087836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704317e-a11f-4848-9907-8ee9ec951931 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.094201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 456/937] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:46:19 2022] GET /identity/v3/endpoints?service_id=b7c8d3120b304c8487d18462e239504d&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:20.100145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.162740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:20.210491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704317e-a11f-4848-9907-8ee9ec951931 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.213994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6f605-84a6-45ab-acb4-1fa42937371d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.252051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.252777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.253061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.253336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.253824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.254123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:20.254432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.254539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.254758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.255029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.255702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.255882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.257066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.263064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 457/938] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:46:20 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:46:20.263836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.275149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 482/939] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:20.280871 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.307175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:20.394868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76116727-7369-4ee9-9c50-6f63d8a1dacb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.423846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:20.452164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973cef-3d70-4ba8-8b74-71d9938570a5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.453481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.456399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.456916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.457476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.458458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.460203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:20.480093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:20.481656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 15:46:20.482582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.486001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 458/940] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Jun 7 15:46:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:20.493062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.499304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f913-1679-4446-812c-c7eda031debd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.500542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f913-1679-4446-812c-c7eda031debd None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.501049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f913-1679-4446-812c-c7eda031debd None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.501530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f913-1679-4446-812c-c7eda031debd None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/d95f13b6e27c43a7868a8ef79621eb7e/roles/125bd519889240d29b02cfd21e83bab8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.504288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f913-1679-4446-812c-c7eda031debd None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=d95f13b6e27c43a7868a8ef79621eb7e, role_id=125bd519889240d29b02cfd21e83bab8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.505911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f913-1679-4446-812c-c7eda031debd None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.523033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 483/941] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:46:20 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/d95f13b6e27c43a7868a8ef79621eb7e/roles/125bd519889240d29b02cfd21e83bab8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:46:20.570267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91612d-646d-4699-9b5d-7d960a87505b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.570574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91612d-646d-4699-9b5d-7d960a87505b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.570854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91612d-646d-4699-9b5d-7d960a87505b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.573785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 484/942] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:20.578327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24327516-6284-4c9c-85cf-9aa3fdd00cb3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.578623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24327516-6284-4c9c-85cf-9aa3fdd00cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.578882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24327516-6284-4c9c-85cf-9aa3fdd00cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.597995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-24327516-6284-4c9c-85cf-9aa3fdd00cb3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:20.599217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-24327516-6284-4c9c-85cf-9aa3fdd00cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:20.640498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e05f56-1af3-4975-810e-fcfacf53e272 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.683979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:20.686863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.687741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.688088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.690322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.690322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:20.690322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.690698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 485/943] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:20.690977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.698518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 459/944] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:20.704230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24327516-6284-4c9c-85cf-9aa3fdd00cb3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.715757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.776303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:20.830497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24327516-6284-4c9c-85cf-9aa3fdd00cb3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.847868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e5095-f2ff-48c0-83be-23942bf3240e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:20.869269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.870213 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.870635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.871050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.871893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:20.872337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.873944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.880233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 486/945] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:20.885535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.891140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:20.891816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:20.892107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:20.892404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:20.893123 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:20.894681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:20.896810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING keystone.server.flask.common [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 15:46:20.922964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.sql.core [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 15:46:20.922964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 15:46:20.922964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [parameters: {'id': 'eff7146f88bc443993ef5aa0fce20df1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 15:46:20.922964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80320) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eff7146f88bc443993ef5aa0fce20df1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eff7146f88bc443993ef5aa0fce20df1. Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 15:46:20.933256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 15:46:20.934152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application yield resource Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise self.value Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application session.commit() Jun 07 15:46:20.939898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.flush() Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application compat.raise_( Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 15:46:20.942147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 15:46:20.943293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 15:46:20.944306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [parameters: {'id': 'eff7146f88bc443993ef5aa0fce20df1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:20.945590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:20.946543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eff7146f88bc443993ef5aa0fce20df1. Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:20.947652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 460/946] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Jun 7 15:46:20 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:20.951324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:20.972215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:21.019841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327e79c-67e0-4d32-aff5-8e4c6e18b404 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:21.064923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:21.066369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:21.067068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:21.067365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:21.068277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:21.069848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:21.091246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:21.095320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab9c15-bdff-4ccb-8dac-c91343de6c2d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:21.099968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 487/947] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:21.155760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fca2e-b3ab-43cc-a2cc-bdb99cd35017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:21.156991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fca2e-b3ab-43cc-a2cc-bdb99cd35017 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:21.157417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fca2e-b3ab-43cc-a2cc-bdb99cd35017 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:21.157863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fca2e-b3ab-43cc-a2cc-bdb99cd35017 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:21.167185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fca2e-b3ab-43cc-a2cc-bdb99cd35017 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:21.169163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fca2e-b3ab-43cc-a2cc-bdb99cd35017 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-419fca2e-b3ab-43cc-a2cc-bdb99cd35017 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:21.185847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:21.186947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:21.188032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:21.190614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 461/948] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:21.196502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:21.313524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52a4f4-d30c-43ff-8a2c-c5fb47b12fe1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:21.333174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:21.352815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b7bc2-902b-4cdf-811b-e3e2b6c52e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:21.353531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b7bc2-902b-4cdf-811b-e3e2b6c52e10 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:21.353813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b7bc2-902b-4cdf-811b-e3e2b6c52e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:21.354082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b7bc2-902b-4cdf-811b-e3e2b6c52e10 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:21.354824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b7bc2-902b-4cdf-811b-e3e2b6c52e10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:21.355126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b7bc2-902b-4cdf-811b-e3e2b6c52e10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:21.356594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b7bc2-902b-4cdf-811b-e3e2b6c52e10 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:21.375877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 488/949] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:22.631150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5602a-9530-4028-b742-c18309e66cf8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:22.631632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5602a-9530-4028-b742-c18309e66cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:22.631632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5602a-9530-4028-b742-c18309e66cf8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:22.633230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 462/950] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:22.638626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd015a43-3e7b-42a2-a135-117398f50c79 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:22.639032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd015a43-3e7b-42a2-a135-117398f50c79 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:22.639220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd015a43-3e7b-42a2-a135-117398f50c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:22.658002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-dd015a43-3e7b-42a2-a135-117398f50c79 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:22.659422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dd015a43-3e7b-42a2-a135-117398f50c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:22.731441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 489/951] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:22.754537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5602a-9530-4028-b742-c18309e66cf8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:22.860928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5602a-9530-4028-b742-c18309e66cf8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:22.896265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4968c7-28ac-46b0-a7d5-715213748673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:22.897178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4968c7-28ac-46b0-a7d5-715213748673 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:22.897459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4968c7-28ac-46b0-a7d5-715213748673 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:22.897731 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4968c7-28ac-46b0-a7d5-715213748673 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:22.898376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4968c7-28ac-46b0-a7d5-715213748673 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:22.899727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4968c7-28ac-46b0-a7d5-715213748673 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ed4968c7-28ac-46b0-a7d5-715213748673 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:22.912257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:22.913528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:22.914423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 463/952] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:22.919760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd015a43-3e7b-42a2-a135-117398f50c79 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:22.940948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86449d-e49b-4ff0-a442-a6aed1963d40 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:22.941432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86449d-e49b-4ff0-a442-a6aed1963d40 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:22.941802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86449d-e49b-4ff0-a442-a6aed1963d40 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:22.943558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 464/953] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:22.948758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270caa3-90c5-4c42-90ce-f85e930a46a9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:22.949042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270caa3-90c5-4c42-90ce-f85e930a46a9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:22.949316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270caa3-90c5-4c42-90ce-f85e930a46a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:22.968092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c270caa3-90c5-4c42-90ce-f85e930a46a9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:22.969756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c270caa3-90c5-4c42-90ce-f85e930a46a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:22.985997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.045893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd015a43-3e7b-42a2-a135-117398f50c79 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.046829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 465/954] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:23.056867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.070726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270caa3-90c5-4c42-90ce-f85e930a46a9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.087793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.089447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.090382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.090681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.091359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:23.091655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:23.093970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:23.102844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 490/955] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:23.108723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.211783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270caa3-90c5-4c42-90ce-f85e930a46a9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.240871 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa7181-782a-4dc7-a78b-b63655f2528f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.249707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.250650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.251022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.251395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.263237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:23.271843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:23.273047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:23.274100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:23.280691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 466/956] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:23.281961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.288465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.289328 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.289866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.290287 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.291153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:23.291644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:23.293741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:23.296843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.307402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 491/957] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:23.308572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.347753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.425836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e489-88e7-41b3-a931-8e9630d1329c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.433224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff17aeb-3cb9-42f1-800c-fabe61af6879 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.464663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.465560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.465935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.466302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.467130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:23.468627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:23.478157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab464d5-41bf-43e1-970f-3b4f86c6b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.479197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab464d5-41bf-43e1-970f-3b4f86c6b38e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.479603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab464d5-41bf-43e1-970f-3b4f86c6b38e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.479964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab464d5-41bf-43e1-970f-3b4f86c6b38e None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.480852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab464d5-41bf-43e1-970f-3b4f86c6b38e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:23.481231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab464d5-41bf-43e1-970f-3b4f86c6b38e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:23.482608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab464d5-41bf-43e1-970f-3b4f86c6b38e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:23.490302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 492/958] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:23.504292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 467/959] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:23.506143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.530185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630225-9bc5-4011-b713-1245008a1802 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.530508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630225-9bc5-4011-b713-1245008a1802 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.530738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630225-9bc5-4011-b713-1245008a1802 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.532301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 468/960] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:23.539437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f09d7-f0ec-4f7d-9b57-1812f4313af5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.539770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f09d7-f0ec-4f7d-9b57-1812f4313af5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.539984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f09d7-f0ec-4f7d-9b57-1812f4313af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.563989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-572f09d7-f0ec-4f7d-9b57-1812f4313af5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:23.566158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-572f09d7-f0ec-4f7d-9b57-1812f4313af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:23.634504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.653321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec0483-d3eb-444e-b40c-68acced4df6d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.660382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 469/961] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:23.681298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f09d7-f0ec-4f7d-9b57-1812f4313af5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.683921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.701715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f7ef1-0fba-4e2a-9af1-fbc7c28d86ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.702658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7ef1-0fba-4e2a-9af1-fbc7c28d86ab None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.702944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7ef1-0fba-4e2a-9af1-fbc7c28d86ab None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.703217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7ef1-0fba-4e2a-9af1-fbc7c28d86ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.716273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f7ef1-0fba-4e2a-9af1-fbc7c28d86ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:23.717759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f7ef1-0fba-4e2a-9af1-fbc7c28d86ab None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-b19f7ef1-0fba-4e2a-9af1-fbc7c28d86ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:23.736239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:23.738243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:23.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:23.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:23.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:23.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:23.739136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 493/962] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:23.742831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f6e6-5ccb-4f45-8e79-ff31be983fcb None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.742831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f6e6-5ccb-4f45-8e79-ff31be983fcb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.743098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f6e6-5ccb-4f45-8e79-ff31be983fcb None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.744573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 494/963] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:23.745148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0f6e6-5ccb-4f45-8e79-ff31be983fcb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.806138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f09d7-f0ec-4f7d-9b57-1812f4313af5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.842800 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de753e-387a-4ed9-9c9e-621b983386cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.843595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de753e-387a-4ed9-9c9e-621b983386cf None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.843877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de753e-387a-4ed9-9c9e-621b983386cf None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.844144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de753e-387a-4ed9-9c9e-621b983386cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.844825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de753e-387a-4ed9-9c9e-621b983386cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:23.846745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de753e-387a-4ed9-9c9e-621b983386cf None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:23.854543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0f6e6-5ccb-4f45-8e79-ff31be983fcb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-10de753e-387a-4ed9-9c9e-621b983386cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:23.862328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:23.863340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:23.864187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:23.864966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 470/964] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:23.866104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4122a2d-b032-4ee7-a109-07de4fcd10cb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.866392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4122a2d-b032-4ee7-a109-07de4fcd10cb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.866792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4122a2d-b032-4ee7-a109-07de4fcd10cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.884752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e4122a2d-b032-4ee7-a109-07de4fcd10cb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:23.885939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e4122a2d-b032-4ee7-a109-07de4fcd10cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:23.895726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:23.896450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:23.896732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:23.897005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:23.897836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:23.898143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:23.899503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:23.908461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.913454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 495/965] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:23.914339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:23.961802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:23.976627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 471/966] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:23.976627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4122a2d-b032-4ee7-a109-07de4fcd10cb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.025486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd706ab-225c-4a6e-ac37-0b01b9290c45 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.062245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.063013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.063297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.063633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.064337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:24.065620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.066665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.076539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 496/967] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:24.076539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.097475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4122a2d-b032-4ee7-a109-07de4fcd10cb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.138280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.139126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.139400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.139670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.140542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:24.147910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.149460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:24.165370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:24.166431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:24.167403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 472/968] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:46:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:24.168930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.191858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ff19-858e-46c8-ab1d-1ae9af6ed26b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.228048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.228665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.228950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.229227 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.229922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.231353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.232671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING keystone.server.flask.common [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 15:46:24.239040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:24.245489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.sql.core [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 15:46:24.245489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 15:46:24.245489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [parameters: {'user_id': 'b3ba2e4f66e142efa8e0e1aefb0b7907', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 15:46:24.245489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80319) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application conn.query(q) Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result.read() Jun 07 15:46:24.261878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 15:46:24.262914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application next(self.gen) Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application yield resource Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application next(self.gen) Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise self.value Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application session.commit() Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 15:46:24.265120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.session.flush() Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application compat.raise_( Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 15:46:24.266011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 15:46:24.266902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application conn.query(q) Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application result.read() Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 15:46:24.268690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b3ba2e4f66e142efa8e0e1aefb0b7907', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:24.269769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 15:46:24.270637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 497/969] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Jun 7 15:46:24 2022] POST /identity/v3/users => generated 160 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:24.272788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.303547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db54fb-5a43-49e6-bfb0-c9113afbcd26 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.339637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.340327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.340622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.340904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.341602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:24.341909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.343336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.349384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 473/970] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:24.362393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.382599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16350be3-8f68-4dab-ae24-46d19a2a562e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.414348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:24.430071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.431735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.432170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.432637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.433979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:24.434482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.437328 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.469232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 498/971] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Jun 7 15:46:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:24.477036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.502508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fd4f5-43f5-4bea-acca-7b4ee9ecadd2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.526610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:24.543691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.544690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.545403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.545548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.553830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.555646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:24.573655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:24.574823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 15:46:24.575989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:24.577830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 474/972] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:46:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:24.583497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.599448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b677dc-9e1d-43b0-9eef-8723dc7a3e3e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.638112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.638959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.639349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.639752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.693447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1e8cd-70e8-4a85-bd8b-49dcf3dfd131 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:24.711671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:24.726473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.728394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.729888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:24.730571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:24.730858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:24.731138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:24.731941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:24.732252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:24.733710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:24.780837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 475/973] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Jun 7 15:46:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:24.843154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 499/974] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:24 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:46:24.853540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:24.901960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:24.972895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30ed88-b85d-47f1-8537-0fc491d20eef None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:25.009654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:25.010457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:25.010743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:25.011023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:25.022256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:25.023771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:25.042254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:25.043305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:25.044665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:25.044665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:25.044665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:25.044665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:25.044665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 476/975] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:25.048966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:25.147198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:25.166018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0281ac2-dfa7-4f6e-9a89-51e201349fb3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:25.202127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:25.202793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:25.203077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:25.203412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:25.204128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:25.204512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:25.206515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:25.218846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 500/976] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:25.224951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:25.318698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:25.329619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d4f0d-f880-4f6f-82a5-0bb14a2e311c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:25.366107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:25.366901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:25.367186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:25.367464 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:25.375571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:25.377051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:25.391261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:25.392190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:25.393600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:25.394740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 477/977] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:25.398321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:25.503487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7c3c8-1ad2-44bf-a4c4-fe097bb2557b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:25.540992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a17fa2-bf87-43ac-adea-633491cbc2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:25.541836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a17fa2-bf87-43ac-adea-633491cbc2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:25.542259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a17fa2-bf87-43ac-adea-633491cbc2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:25.542641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a17fa2-bf87-43ac-adea-633491cbc2e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:25.543530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a17fa2-bf87-43ac-adea-633491cbc2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:25.543946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a17fa2-bf87-43ac-adea-633491cbc2e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:25.545512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a17fa2-bf87-43ac-adea-633491cbc2e5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:25.562720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 501/978] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:25.563999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:25.667669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bfd43-7c16-48fc-b88d-4f1e4d03f46d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:25.702522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e73a5-1be8-4f60-a842-027f6b5809e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:25.703298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e73a5-1be8-4f60-a842-027f6b5809e7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:25.703486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e73a5-1be8-4f60-a842-027f6b5809e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:25.703728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e73a5-1be8-4f60-a842-027f6b5809e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:25.704580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e73a5-1be8-4f60-a842-027f6b5809e7 None admin] RBAC: Adding query filter params (role.id=125bd519889240d29b02cfd21e83bab8, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=d95f13b6e27c43a7868a8ef79621eb7e) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:25.704895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e73a5-1be8-4f60-a842-027f6b5809e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:25.706586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e73a5-1be8-4f60-a842-027f6b5809e7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:25.720645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 478/979] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:25 2022] GET /identity/v3/role_assignments?user.id=d95f13b6e27c43a7868a8ef79621eb7e&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=125bd519889240d29b02cfd21e83bab8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:25.851512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0722e-2ad9-42e6-a0ee-e464e99d8d9b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:25.852068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0722e-2ad9-42e6-a0ee-e464e99d8d9b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:25.852068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0722e-2ad9-42e6-a0ee-e464e99d8d9b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:25.853749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 502/980] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:25.859337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e3d74-3349-4cff-acce-69f02b0ae70a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:25.859651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e3d74-3349-4cff-acce-69f02b0ae70a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:25.859901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e3d74-3349-4cff-acce-69f02b0ae70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:25.878764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ef6e3d74-3349-4cff-acce-69f02b0ae70a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:25.879983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef6e3d74-3349-4cff-acce-69f02b0ae70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:25.891286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:25.956538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 479/981] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:25.978752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0722e-2ad9-42e6-a0ee-e464e99d8d9b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:25.982658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:26.085234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0722e-2ad9-42e6-a0ee-e464e99d8d9b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:26.120502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:26.121301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:26.121576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:26.121840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:26.122477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:26.123819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:26.137010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:26.138062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:26.138913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 503/982] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:26.149734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e3d74-3349-4cff-acce-69f02b0ae70a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:26.251795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e3d74-3349-4cff-acce-69f02b0ae70a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:26.286616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:26.287285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:26.287563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:26.287835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:26.288556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:26.288865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:26.290234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:26.300271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 480/983] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:26.301470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:26.369205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:26.408273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6952f8b-ae56-4d38-ab93-0767c8f9759c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:26.445319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:26.445981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:26.446260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:26.446532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:26.447229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:26.447527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:26.448929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:26.454165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 504/984] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:26.460498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:26.472252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:26.564435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc636ac-9e30-432f-bad0-1cd44eb7bc38 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:26.601258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f1976-5d3b-482a-b597-c16fa881a5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:26.601922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f1976-5d3b-482a-b597-c16fa881a5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:26.602202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f1976-5d3b-482a-b597-c16fa881a5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:26.602474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f1976-5d3b-482a-b597-c16fa881a5e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:26.603258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f1976-5d3b-482a-b597-c16fa881a5e4 None admin] RBAC: Adding query filter params (interface=admin, service_id=b7c8d3120b304c8487d18462e239504d, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:26.603568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f1976-5d3b-482a-b597-c16fa881a5e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:26.604983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f1976-5d3b-482a-b597-c16fa881a5e4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:26.615349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 481/985] 10.222.0.18 () {56 vars in 1249 bytes} [Tue Jun 7 15:46:26 2022] GET /identity/v3/endpoints?service_id=b7c8d3120b304c8487d18462e239504d&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:26.616316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:26.716113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6781069-56d0-4097-ab17-067c2f305067 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:26.746381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:26.752721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46f8f5-ca66-42e3-9e87-fa6574172b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:26.753391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f8f5-ca66-42e3-9e87-fa6574172b51 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:26.753675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f8f5-ca66-42e3-9e87-fa6574172b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:26.753953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f8f5-ca66-42e3-9e87-fa6574172b51 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:26.754635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46f8f5-ca66-42e3-9e87-fa6574172b51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:26.756012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46f8f5-ca66-42e3-9e87-fa6574172b51 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:26.761635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7583-ac1e-49aa-a77a-cfd87d94d95d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:26.762097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7583-ac1e-49aa-a77a-cfd87d94d95d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:26.762173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d7583-ac1e-49aa-a77a-cfd87d94d95d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:26.768481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 505/986] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:46:26 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:46:26.768652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 482/987] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:26.769425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cdfb0-2fbe-43ce-b9d4-268797876154 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:26.769716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cdfb0-2fbe-43ce-b9d4-268797876154 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:26.769995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cdfb0-2fbe-43ce-b9d4-268797876154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:26.789078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e13cdfb0-2fbe-43ce-b9d4-268797876154 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:26.790653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e13cdfb0-2fbe-43ce-b9d4-268797876154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:26.869782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 506/988] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:26.893812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d7583-ac1e-49aa-a77a-cfd87d94d95d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:26.974997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:27.007262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d7583-ac1e-49aa-a77a-cfd87d94d95d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:27.041260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65eae8-619c-4eb7-b249-cab163445daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:27.041926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65eae8-619c-4eb7-b249-cab163445daa None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:27.042206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65eae8-619c-4eb7-b249-cab163445daa None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:27.042478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65eae8-619c-4eb7-b249-cab163445daa None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:27.043196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65eae8-619c-4eb7-b249-cab163445daa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:27.044726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65eae8-619c-4eb7-b249-cab163445daa None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:27.064393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 483/989] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:46:26 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:28.017356 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326220d-0d35-4d6a-95b5-e885000ffa67 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:28.017879 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326220d-0d35-4d6a-95b5-e885000ffa67 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:28.018015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326220d-0d35-4d6a-95b5-e885000ffa67 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:28.019454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 507/990] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:28.024559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda2f88-aa9e-4f1f-8f05-126ec8a55a05 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:28.024938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda2f88-aa9e-4f1f-8f05-126ec8a55a05 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:28.025120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda2f88-aa9e-4f1f-8f05-126ec8a55a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:28.042027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ceda2f88-aa9e-4f1f-8f05-126ec8a55a05 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:28.043313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ceda2f88-aa9e-4f1f-8f05-126ec8a55a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:28.117044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 484/991] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:28.139178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326220d-0d35-4d6a-95b5-e885000ffa67 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:28.244417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326220d-0d35-4d6a-95b5-e885000ffa67 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:28.272282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:28.281698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:28.282488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:28.282765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:28.283033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] PATH_INFO: `/v3/services/image` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:28.284192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:28.285575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:28.297201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:28.298183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:28.299019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 508/992] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Jun 7 15:46:28 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:28.304910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda2f88-aa9e-4f1f-8f05-126ec8a55a05 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:28.364898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:28.409258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda2f88-aa9e-4f1f-8f05-126ec8a55a05 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:28.444735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45db001-c05f-40d4-ac40-d15eedda8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:28.445410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db001-c05f-40d4-ac40-d15eedda8931 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:28.445693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db001-c05f-40d4-ac40-d15eedda8931 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:28.445964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db001-c05f-40d4-ac40-d15eedda8931 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:28.446674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45db001-c05f-40d4-ac40-d15eedda8931 None admin] RBAC: Adding query filter params (name=image) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:28.446979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45db001-c05f-40d4-ac40-d15eedda8931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:28.448338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45db001-c05f-40d4-ac40-d15eedda8931 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:28.459024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 485/993] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:28.460206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:28.561049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d55932-d94a-4e08-bf5c-9bf71edfd867 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:28.596262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75b2de-441e-41d4-921b-4076c3f63bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:28.596956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b2de-441e-41d4-921b-4076c3f63bba None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:28.597238 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b2de-441e-41d4-921b-4076c3f63bba None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:28.597510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b2de-441e-41d4-921b-4076c3f63bba None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:28.598208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75b2de-441e-41d4-921b-4076c3f63bba None admin] RBAC: Adding query filter params (type=image) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:28.598510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75b2de-441e-41d4-921b-4076c3f63bba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:28.600116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75b2de-441e-41d4-921b-4076c3f63bba None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:28.604900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 509/994] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:29.071580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f05f0-d9b9-4d17-a793-d5ae93616922 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.072100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f05f0-d9b9-4d17-a793-d5ae93616922 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.072177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f05f0-d9b9-4d17-a793-d5ae93616922 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.073543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 486/995] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:29.078835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a57a8-0070-4b49-b2b3-5bc7ac8b4c6b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.079232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a57a8-0070-4b49-b2b3-5bc7ac8b4c6b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.079379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a57a8-0070-4b49-b2b3-5bc7ac8b4c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.097635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-812a57a8-0070-4b49-b2b3-5bc7ac8b4c6b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:29.098804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-812a57a8-0070-4b49-b2b3-5bc7ac8b4c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:29.156090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:29.175343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 510/996] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:29.197810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f05f0-d9b9-4d17-a793-d5ae93616922 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:29.301692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f05f0-d9b9-4d17-a793-d5ae93616922 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:29.324724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:29.337495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed43a2-6b1d-41bb-aabf-8fe4ce58d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:29.338301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed43a2-6b1d-41bb-aabf-8fe4ce58d180 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.338576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed43a2-6b1d-41bb-aabf-8fe4ce58d180 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.338841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed43a2-6b1d-41bb-aabf-8fe4ce58d180 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.339481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed43a2-6b1d-41bb-aabf-8fe4ce58d180 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:29.340855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed43a2-6b1d-41bb-aabf-8fe4ce58d180 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-3aed43a2-6b1d-41bb-aabf-8fe4ce58d180 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:29.353811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:29.354807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 15:46:29.355643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:29.359996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 487/997] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Jun 7 15:46:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:29.361242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a57a8-0070-4b49-b2b3-5bc7ac8b4c6b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:29.362737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745fb97-749b-47e9-a42a-d98d5bee95b7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.363195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745fb97-749b-47e9-a42a-d98d5bee95b7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.363476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745fb97-749b-47e9-a42a-d98d5bee95b7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.365063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 488/998] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:29.369869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d7207-0d96-4e0e-bd74-b0f28e060311 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.370156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d7207-0d96-4e0e-bd74-b0f28e060311 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.370431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d7207-0d96-4e0e-bd74-b0f28e060311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.389567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8a2d7207-0d96-4e0e-bd74-b0f28e060311 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:29.390742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8a2d7207-0d96-4e0e-bd74-b0f28e060311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:29.464852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 489/999] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:29.472938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a57a8-0070-4b49-b2b3-5bc7ac8b4c6b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:29.488566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d7207-0d96-4e0e-bd74-b0f28e060311 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:29.511334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:29.512013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.512297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.512608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.513307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:29.513618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:29.514993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:29.520842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 511/1000] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:29.526170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:29.577498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:29.595429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d7207-0d96-4e0e-bd74-b0f28e060311 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:29.614974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:29.633280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:29.634075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.634362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.634641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.635434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:29.636847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:29.638363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1599ec-9be2-4795-999d-c6f58947c875 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:29.650697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:29.651787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:29.656970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 490/1001] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:29.659125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:29.676486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:29.677149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.677434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.677711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.678415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:29.678718 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:29.680071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:29.685059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 512/1002] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Jun 7 15:46:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:29.690556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:29.782133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804e8f1-f743-4fb9-9147-9ce4f0ee4ef9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:29.804444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349641d-2a86-4631-a675-10af5ff19980 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:29.818556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:29.819227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.819512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.819788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.820520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:29.820829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:29.822182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:29.822293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:29.827821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 491/1003] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:29.836000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:29.851045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:29.851715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.851994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:29.852274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:29.853050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:29.854488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:29.876788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 513/1004] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:46:29.888753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:29.954436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1413b7-02fe-4193-b44b-ed745e800284 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:29.998806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:29.999575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:29.999927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:30.000268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:30.001056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:30.001429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:30.003517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:30.013958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 492/1005] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:30.015765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:30.123735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de71c3-b54b-4c72-8af0-bafd6013a222 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:30.162724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90fcee-29d5-429d-b123-e1309939019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:30.163393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90fcee-29d5-429d-b123-e1309939019c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:30.163669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90fcee-29d5-429d-b123-e1309939019c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:30.163938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90fcee-29d5-429d-b123-e1309939019c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:30.164852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90fcee-29d5-429d-b123-e1309939019c None admin] RBAC: Adding query filter params (interface=public, service_id=82ed10dda1324c0292bd429a932f863a, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:30.165159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90fcee-29d5-429d-b123-e1309939019c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:30.166510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90fcee-29d5-429d-b123-e1309939019c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:30.174549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 514/1006] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:46:30 2022] GET /identity/v3/endpoints?service_id=82ed10dda1324c0292bd429a932f863a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:30.177564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:30.282662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3bc31-6fa6-44e8-a866-edbd33525b8e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:30.285466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:30.319166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd64f8-6cc8-4cf7-af54-2dacd7b1bfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:30.319833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd64f8-6cc8-4cf7-af54-2dacd7b1bfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:30.320113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd64f8-6cc8-4cf7-af54-2dacd7b1bfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:30.320580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd64f8-6cc8-4cf7-af54-2dacd7b1bfa8 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:30.321103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd64f8-6cc8-4cf7-af54-2dacd7b1bfa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:30.322446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd64f8-6cc8-4cf7-af54-2dacd7b1bfa8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:30.335259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 493/1007] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:46:30 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:46:30.888925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d52fe-1234-4fc0-9ec2-be517dc5e108 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:30.889304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d52fe-1234-4fc0-9ec2-be517dc5e108 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:30.889485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d52fe-1234-4fc0-9ec2-be517dc5e108 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:30.891012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 515/1008] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:30.896617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5f631-1419-46a1-961e-e75d5600ecbc None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:30.896897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5f631-1419-46a1-961e-e75d5600ecbc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:30.897174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5f631-1419-46a1-961e-e75d5600ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:30.914029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1ad5f631-1419-46a1-961e-e75d5600ecbc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:30.915203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1ad5f631-1419-46a1-961e-e75d5600ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:30.981717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 494/1009] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:31.005397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d52fe-1234-4fc0-9ec2-be517dc5e108 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:31.043930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:31.116762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d52fe-1234-4fc0-9ec2-be517dc5e108 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:31.158082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3fbe-5c57-4cfb-b84b-3077697f1f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:31.158940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3fbe-5c57-4cfb-b84b-3077697f1f5a None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:31.159193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3fbe-5c57-4cfb-b84b-3077697f1f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:31.159642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3fbe-5c57-4cfb-b84b-3077697f1f5a None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:31.160411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3fbe-5c57-4cfb-b84b-3077697f1f5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:31.161776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3fbe-5c57-4cfb-b84b-3077697f1f5a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:31.177809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 516/1010] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:46:31 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:32.192004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8e1e-3bdc-4d58-b757-6ca4498e8857 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:32.192912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8e1e-3bdc-4d58-b757-6ca4498e8857 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:32.193199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8e1e-3bdc-4d58-b757-6ca4498e8857 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:32.194605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 495/1011] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:32.199907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b6766-51b2-4c10-a7b5-0d35ae11872c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:32.200323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b6766-51b2-4c10-a7b5-0d35ae11872c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:32.200536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b6766-51b2-4c10-a7b5-0d35ae11872c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:32.217469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f89b6766-51b2-4c10-a7b5-0d35ae11872c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:32.218644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f89b6766-51b2-4c10-a7b5-0d35ae11872c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:32.297789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 517/1012] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:32 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:32.322273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8e1e-3bdc-4d58-b757-6ca4498e8857 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:32.427400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:32.432539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8e1e-3bdc-4d58-b757-6ca4498e8857 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:32.470433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eebc73-adf0-496f-b9f9-c7dca5e20a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:32.471697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebc73-adf0-496f-b9f9-c7dca5e20a50 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:32.471697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebc73-adf0-496f-b9f9-c7dca5e20a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:32.471953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebc73-adf0-496f-b9f9-c7dca5e20a50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:32.473115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eebc73-adf0-496f-b9f9-c7dca5e20a50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:32.474806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eebc73-adf0-496f-b9f9-c7dca5e20a50 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:32.476615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 496/1013] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:32.482092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b6766-51b2-4c10-a7b5-0d35ae11872c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:32.507610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:32.583560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a45c1-94e2-4721-bfd2-9cb51268530b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:32.583838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a45c1-94e2-4721-bfd2-9cb51268530b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:32.584319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a45c1-94e2-4721-bfd2-9cb51268530b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:32.585840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 497/1014] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:32.586123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b6766-51b2-4c10-a7b5-0d35ae11872c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:32.590697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007ef15-90e4-44a6-958a-00ad71e2533f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:32.590980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007ef15-90e4-44a6-958a-00ad71e2533f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:32.591255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007ef15-90e4-44a6-958a-00ad71e2533f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:32.608280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2007ef15-90e4-44a6-958a-00ad71e2533f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:32.609461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2007ef15-90e4-44a6-958a-00ad71e2533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:32.622247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:32.622899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:32.623178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:32.623448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:32.624159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:32.626262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:32.652755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 518/1015] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:46:32 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:32.687839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 498/1016] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:32 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:32.712003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:32.795627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:32.825476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8367c-bc30-41ca-b682-8cc71a0e3203 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:32.861704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:32.862565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:32.862799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:32.863067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:32.863720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:32.865129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:32.877008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:32.879263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:32.880429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 519/1017] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:32.886346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007ef15-90e4-44a6-958a-00ad71e2533f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:32.983204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:32.989893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007ef15-90e4-44a6-958a-00ad71e2533f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:33.024752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:33.025419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:33.025699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:33.025975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:33.026669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:33.026972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:33.028402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:33.038516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 499/1018] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:33.039784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:33.145423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae5b0-eda4-41ad-8f79-b656957f0cf7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:33.185751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032df5b-2345-4132-a7e0-88049e81e4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:33.186414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032df5b-2345-4132-a7e0-88049e81e4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:33.186693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032df5b-2345-4132-a7e0-88049e81e4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:33.186964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032df5b-2345-4132-a7e0-88049e81e4b0 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:33.187667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032df5b-2345-4132-a7e0-88049e81e4b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:33.187971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032df5b-2345-4132-a7e0-88049e81e4b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:33.189352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032df5b-2345-4132-a7e0-88049e81e4b0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:33.199215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 520/1019] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Jun 7 15:46:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:33.200322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:33.303218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f207a-97a4-4738-866d-99100f5e70c3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:33.337775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462dc017-de23-4a1e-8949-4bfcced9f309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:33.338439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dc017-de23-4a1e-8949-4bfcced9f309 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:33.338720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dc017-de23-4a1e-8949-4bfcced9f309 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:33.338992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dc017-de23-4a1e-8949-4bfcced9f309 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:33.339725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dc017-de23-4a1e-8949-4bfcced9f309 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:33.341109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dc017-de23-4a1e-8949-4bfcced9f309 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:33.368764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 500/1020] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:33.493422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185f816-ea95-435c-b709-23d54ba7aab5 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:33.494318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185f816-ea95-435c-b709-23d54ba7aab5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:33.494603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185f816-ea95-435c-b709-23d54ba7aab5 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:33.496204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 521/1021] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:33.501857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5521d8-51e4-4599-8c06-d54c49454ee0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:33.502147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5521d8-51e4-4599-8c06-d54c49454ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:33.502995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5521d8-51e4-4599-8c06-d54c49454ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:33.521092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9e5521d8-51e4-4599-8c06-d54c49454ee0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:33.522324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9e5521d8-51e4-4599-8c06-d54c49454ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:33.535693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:33.604657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 501/1022] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:33.627907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185f816-ea95-435c-b709-23d54ba7aab5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:33.630946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:33.739585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185f816-ea95-435c-b709-23d54ba7aab5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:33.778215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:33.779311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:33.779772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:33.780259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:33.781120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:33.782687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:33.795612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:33.796681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:33.797510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:33.798246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 522/1023] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Jun 7 15:46:33 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:33.804238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5521d8-51e4-4599-8c06-d54c49454ee0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:33.915130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5521d8-51e4-4599-8c06-d54c49454ee0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:33.951817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:33.952517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:33.952800 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:33.953098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:33.953772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] RBAC: Adding query filter params (name=image) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:33.954075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:33.955987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:33.961454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 502/1024] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:33.966949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:34.039030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:34.082325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88e8de-ce2b-4ac0-a6d7-291c53ac35c6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:34.121763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:34.122428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:34.122712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:34.122989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:34.123694 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] RBAC: Adding query filter params (type=image) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:34.124005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:34.125390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:34.134272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 523/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:34.140223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:34.149837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:34.245934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf193-81c7-4c62-ac1e-51ca4bf7c224 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:34.281230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193af6-9293-4572-a20e-d0362df1d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:34.282124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193af6-9293-4572-a20e-d0362df1d3be None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:34.282554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193af6-9293-4572-a20e-d0362df1d3be None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:34.282996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193af6-9293-4572-a20e-d0362df1d3be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:34.283959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193af6-9293-4572-a20e-d0362df1d3be None admin] RBAC: Adding query filter params (interface=public, service_id=caf318fe54b448bd8e831bc45ae7bcdd, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:34.284613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193af6-9293-4572-a20e-d0362df1d3be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:34.286127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193af6-9293-4572-a20e-d0362df1d3be None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:34.291513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 503/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:46:34 2022] GET /identity/v3/endpoints?service_id=caf318fe54b448bd8e831bc45ae7bcdd&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:34.297107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:34.403233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07f912-e710-4264-a5e2-25d0de395e82 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:34.432864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:34.440593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd49f8-0bb5-40da-be62-1eb4d6a23cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:34.441441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd49f8-0bb5-40da-be62-1eb4d6a23cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:34.441870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd49f8-0bb5-40da-be62-1eb4d6a23cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:34.442323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd49f8-0bb5-40da-be62-1eb4d6a23cc7 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:34.443141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd49f8-0bb5-40da-be62-1eb4d6a23cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:34.444656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd49f8-0bb5-40da-be62-1eb4d6a23cc7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:34.451802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 524/1027] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:46:34 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:46:35.044613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc9844-d218-4df6-862b-8a0031bcbed7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:35.045016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc9844-d218-4df6-862b-8a0031bcbed7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:35.045194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc9844-d218-4df6-862b-8a0031bcbed7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:35.046695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 504/1028] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:35.052103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcf7cf-3f8c-4ba5-9c4c-57deb3ef45d9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:35.052572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcf7cf-3f8c-4ba5-9c4c-57deb3ef45d9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:35.052745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcf7cf-3f8c-4ba5-9c4c-57deb3ef45d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:35.070032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a4dcf7cf-3f8c-4ba5-9c4c-57deb3ef45d9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:35.071207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a4dcf7cf-3f8c-4ba5-9c4c-57deb3ef45d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:35.156767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 525/1029] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:35.181077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc9844-d218-4df6-862b-8a0031bcbed7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:35.255342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:35.289203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc9844-d218-4df6-862b-8a0031bcbed7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:35.324443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5397f8-931d-4763-a52f-d6715349b658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:35.325150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5397f8-931d-4763-a52f-d6715349b658 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:35.325431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5397f8-931d-4763-a52f-d6715349b658 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:35.325704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5397f8-931d-4763-a52f-d6715349b658 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:35.326860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5397f8-931d-4763-a52f-d6715349b658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:35.328276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5397f8-931d-4763-a52f-d6715349b658 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:35.330087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 505/1030] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:35.336408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcf7cf-3f8c-4ba5-9c4c-57deb3ef45d9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:35.442600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcf7cf-3f8c-4ba5-9c4c-57deb3ef45d9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:35.468165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:35.481325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc901b7f-be86-49f3-b895-46b22afb2184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:35.481997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc901b7f-be86-49f3-b895-46b22afb2184 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:35.482278 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc901b7f-be86-49f3-b895-46b22afb2184 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:35.482549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc901b7f-be86-49f3-b895-46b22afb2184 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:35.483266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc901b7f-be86-49f3-b895-46b22afb2184 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:35.484716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc901b7f-be86-49f3-b895-46b22afb2184 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:35.511099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 526/1031] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:46:35 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:35.770100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3547-6456-44cb-b034-704e95937928 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:35.770388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3547-6456-44cb-b034-704e95937928 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:35.770668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3547-6456-44cb-b034-704e95937928 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:35.772320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 506/1032] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:35.777647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98503a1-6606-429c-a7fe-595ecdade1fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:35.778106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98503a1-6606-429c-a7fe-595ecdade1fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:35.778386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98503a1-6606-429c-a7fe-595ecdade1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:35.795860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a98503a1-6606-429c-a7fe-595ecdade1fe None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:35.797070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a98503a1-6606-429c-a7fe-595ecdade1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:35.867242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 527/1033] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:35.892158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f3547-6456-44cb-b034-704e95937928 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:35.997523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f3547-6456-44cb-b034-704e95937928 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:36.032254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:36.033015 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.033337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.033610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.040549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:36.044840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:36.053528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:36.054584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:36.055571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:36.056527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 507/1034] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Jun 7 15:46:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:36.061032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98503a1-6606-429c-a7fe-595ecdade1fe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:36.171538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98503a1-6606-429c-a7fe-595ecdade1fe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:36.183955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:36.210853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:36.211523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.211802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.212075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.212847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:36.213152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:36.214503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:36.222925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 528/1035] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Jun 7 15:46:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:36.228586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:36.330732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f7ad-c698-4807-8d84-0e9b05106b12 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:36.364918 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:36.365740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.366024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.366295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.377448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:36.379175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:36.397628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:36.398592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:36.399493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:36.399493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:36.399493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:36.399493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:36.400555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 508/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:36.404767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:36.512257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c7d2-d907-4371-a1e0-241f1cbcf9a8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:36.546310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:36.546969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.547253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.547532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.548245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:36.548592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:36.550722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:36.567257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 529/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:36.568582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:36.621934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:36.673355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d847c4-70fa-4643-b8fa-bdeec2cb77a2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:36.707447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70b7d8-0b40-4a39-a00d-27e6ba412ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:36.708272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70b7d8-0b40-4a39-a00d-27e6ba412ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.708585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70b7d8-0b40-4a39-a00d-27e6ba412ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.708860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70b7d8-0b40-4a39-a00d-27e6ba412ae5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.717550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70b7d8-0b40-4a39-a00d-27e6ba412ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:36.719001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70b7d8-0b40-4a39-a00d-27e6ba412ae5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6f70b7d8-0b40-4a39-a00d-27e6ba412ae5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:36.734339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:36.735338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:36.736337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:36.740600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 509/1038] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:46:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:36.742038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:36.750339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8167be-edd4-4141-8380-98646ad57264 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.750629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8167be-edd4-4141-8380-98646ad57264 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.750908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8167be-edd4-4141-8380-98646ad57264 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.755307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 510/1039] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:36.757466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a786-4c89-47b5-af67-c05ec20dc474 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.757757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a786-4c89-47b5-af67-c05ec20dc474 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.758027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a786-4c89-47b5-af67-c05ec20dc474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.774969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6292a786-4c89-47b5-af67-c05ec20dc474 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:36.776145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6292a786-4c89-47b5-af67-c05ec20dc474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:36.777885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:36.850945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 511/1040] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:36.854237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3ec7-6f7f-4cd0-9364-3b8c28a0ffca None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:36.873353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a786-4c89-47b5-af67-c05ec20dc474 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:36.889583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:36.890263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:36.890548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:36.890830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:36.891667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:36.891980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:36.894361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:36.915284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:36.917300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 530/1041] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:46:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:36.922378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:36.984824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a786-4c89-47b5-af67-c05ec20dc474 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.020000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:37.022923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.023829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.024121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.024429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.025093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.026447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.030315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498bff7-1281-498f-8f4a-15afc8d767af None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:37.038418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:37.039411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:37.040700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 512/1042] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Jun 7 15:46:36 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:37.045433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:37.071794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.072564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.072851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.073125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.148626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.150450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.169987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10f76f-a130-4b0a-86ae-fe2124e414d9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.173589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:37.215907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.216620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.216907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.217193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.217921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:37.218208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.219579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.232735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 513/1043] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:37.234991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:37.251079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:37.263155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 531/1044] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:46:37.269583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:37.350714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96af11-5b41-449d-b3f8-6d64dc1bfab2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.374769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382dcf7-0f56-4304-bb88-a470ac31271c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.390608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.391269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.391551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.391822 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.392548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] RBAC: Adding query filter params (type=image) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:37.395267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.396672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.406414 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 514/1045] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:37.407765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:37.415794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.416616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.416897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.417169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.429109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.430558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.441504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:37.451551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:37.452772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:37.453832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:37.453832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:37.453832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:37.453832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:37.453832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 532/1046] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:37.459070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:37.512348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:37.515159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efdbce-2b45-41b6-a983-64c95fcb2145 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.551874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.552564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.552850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.553128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.553866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.555231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.576404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 515/1047] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:46:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:46:37.603066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06f2db-65e4-42dc-9150-ebc46169beb8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.651966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bdbc8-a0fb-4c3d-b849-eac93d74588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.653107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bdbc8-a0fb-4c3d-b849-eac93d74588f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.653611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bdbc8-a0fb-4c3d-b849-eac93d74588f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.654083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bdbc8-a0fb-4c3d-b849-eac93d74588f None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.655027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bdbc8-a0fb-4c3d-b849-eac93d74588f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:37.655554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bdbc8-a0fb-4c3d-b849-eac93d74588f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.657333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bdbc8-a0fb-4c3d-b849-eac93d74588f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.675706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 533/1048] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:37.677478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:37.776730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:37.797049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9b6da-7597-47da-89dd-d9943a19ccc7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.831059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:37.831860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.832142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.832447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.839551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:37.841030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:37.856927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:37.857859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:37.858780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 516/1049] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:37.861119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ff8cc-c9c3-4e4b-9c54-881dc2085ce7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.861915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ff8cc-c9c3-4e4b-9c54-881dc2085ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.862370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ff8cc-c9c3-4e4b-9c54-881dc2085ce7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.863934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 534/1050] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:37.864501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:37.869105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75835-32e0-4d82-8cff-98bc3fcce594 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:37.869616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75835-32e0-4d82-8cff-98bc3fcce594 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:37.870103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75835-32e0-4d82-8cff-98bc3fcce594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:37.886685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cbb75835-32e0-4d82-8cff-98bc3fcce594 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:37.887949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cbb75835-32e0-4d82-8cff-98bc3fcce594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:37.963997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 535/1051] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:37.973385 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dd533-67a0-44e5-b43c-1fda6a361e94 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:37.989110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb75835-32e0-4d82-8cff-98bc3fcce594 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:38.010629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:38.011293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:38.011572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:38.011848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:38.012644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:38.012951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:38.014364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:38.017406 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:38.027720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 517/1052] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:38.034044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:38.045469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:38.100185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb75835-32e0-4d82-8cff-98bc3fcce594 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:38.139078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ab6ff-6bb2-4528-b94f-f790647c9239 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:38.140012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:38.140849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:38.141132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:38.141420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:38.155413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:38.163912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:38.164970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:38.169115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 536/1053] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:38.171240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:38.181276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:38.181941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:38.182225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:38.182502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:38.183323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] RBAC: Adding query filter params (role.id=125bd519889240d29b02cfd21e83bab8, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0b352e6a51fc4d1f8b9b97d092d11bd7) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:38.183637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:38.185105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:38.191712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 518/1054] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:38 2022] GET /identity/v3/role_assignments?user.id=0b352e6a51fc4d1f8b9b97d092d11bd7&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=125bd519889240d29b02cfd21e83bab8 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:38.274113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:38.288510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f335209-2f7c-4835-b5d7-cc5dbac2c97c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:38.328042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:38.329124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:38.329603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:38.330121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:38.331674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:38.332484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:38.334101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:38.351096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 537/1055] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:46:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:38.357497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:38.453728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:38.488028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abcb7d-b692-4219-aedb-56605514e606 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:38.523091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:38.523968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:38.524317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:38.524567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] PATH_INFO: `/v3/users/a6ecf8652d3e433296aa4e89a8bf803e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:38.539708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] RBAC: Authorizing `identity:get_user(user_id=a6ecf8652d3e433296aa4e89a8bf803e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:38.541542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:38.543819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 519/1056] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:38 2022] GET /identity/v3/users/a6ecf8652d3e433296aa4e89a8bf803e => generated 285 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:38.550000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:38.654509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b78ea-2264-4979-b1c0-7147617c3182 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:38.695197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:38.695975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:38.696321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:38.696839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:38.719231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:38.778714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:38.780069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:38.878396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 538/1057] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:46:38.883959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:38.988293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfed219-0f09-4875-90cc-9f97c328b6c7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:39.016393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:39.023260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111d47f-c495-4daf-9b64-bc0a1d94a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:39.024119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111d47f-c495-4daf-9b64-bc0a1d94a359 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:39.024450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111d47f-c495-4daf-9b64-bc0a1d94a359 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:39.024727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111d47f-c495-4daf-9b64-bc0a1d94a359 None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:39.025885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111d47f-c495-4daf-9b64-bc0a1d94a359 None admin] RBAC: Authorizing `identity:get_project(project_id=b70c79e5aef14caaa1b106d33f39f76d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:39.027311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111d47f-c495-4daf-9b64-bc0a1d94a359 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:39.034481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 520/1058] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:46:38 2022] GET /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:39.036127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:39.086773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:39.147429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8f210-d3a9-4855-a8a4-6766e3b3b66e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:39.182068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317b3ef-e3b9-4549-9a82-b0ad147e4370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:39.182732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b3ef-e3b9-4549-9a82-b0ad147e4370 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:39.183014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b3ef-e3b9-4549-9a82-b0ad147e4370 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:39.183288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b3ef-e3b9-4549-9a82-b0ad147e4370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:39.184104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b3ef-e3b9-4549-9a82-b0ad147e4370 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=b70c79e5aef14caaa1b106d33f39f76d, user.id=a6ecf8652d3e433296aa4e89a8bf803e) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:39.184434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b3ef-e3b9-4549-9a82-b0ad147e4370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:39.185856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b3ef-e3b9-4549-9a82-b0ad147e4370 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:39.191859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 539/1059] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:39 2022] GET /identity/v3/role_assignments?user.id=a6ecf8652d3e433296aa4e89a8bf803e&scope.project.id=b70c79e5aef14caaa1b106d33f39f76d&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:39.918156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bf601-1eab-44ea-a98a-59366369edd6 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:39.918710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bf601-1eab-44ea-a98a-59366369edd6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:39.918710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bf601-1eab-44ea-a98a-59366369edd6 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:39.920083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 521/1060] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:39.926185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7cb5a-ac66-4e75-be99-8b4fb0f46687 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:39.926470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7cb5a-ac66-4e75-be99-8b4fb0f46687 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:39.926743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7cb5a-ac66-4e75-be99-8b4fb0f46687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:39.945098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-abb7cb5a-ac66-4e75-be99-8b4fb0f46687 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:39.946282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-abb7cb5a-ac66-4e75-be99-8b4fb0f46687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:40.015688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 540/1061] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:40.038404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bf601-1eab-44ea-a98a-59366369edd6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:40.144618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bf601-1eab-44ea-a98a-59366369edd6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:40.183662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86c8f3-47fc-4999-97e2-cce775029bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:40.184227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86c8f3-47fc-4999-97e2-cce775029bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:40.184538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86c8f3-47fc-4999-97e2-cce775029bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:40.184808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86c8f3-47fc-4999-97e2-cce775029bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:40.250243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:40.255441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86c8f3-47fc-4999-97e2-cce775029bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:40.256919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86c8f3-47fc-4999-97e2-cce775029bb9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:40.359891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 522/1062] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:46:40.366734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7cb5a-ac66-4e75-be99-8b4fb0f46687 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:40.412276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:40.467000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7cb5a-ac66-4e75-be99-8b4fb0f46687 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:40.480252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b494b-c952-4d58-80e2-d0f10753f4ff None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:40.480622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b494b-c952-4d58-80e2-d0f10753f4ff None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:40.480820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b494b-c952-4d58-80e2-d0f10753f4ff None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:40.482194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 523/1063] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:46:40.488272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d30bd0-1488-44a7-8e6f-6cf47bc4b7dd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:40.488658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d30bd0-1488-44a7-8e6f-6cf47bc4b7dd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:40.488968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d30bd0-1488-44a7-8e6f-6cf47bc4b7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:40.505112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:40.505903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:40.506184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:40.506454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:40.507169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a2d30bd0-1488-44a7-8e6f-6cf47bc4b7dd None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:40.508380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a2d30bd0-1488-44a7-8e6f-6cf47bc4b7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:40.516659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:40.518816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:40.536137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:40.537054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:40.537942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:40.537942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:40.537942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:40.537942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:40.537942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 541/1064] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:40.543014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:40.585390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 524/1065] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:40.608600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d30bd0-1488-44a7-8e6f-6cf47bc4b7dd None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:40.636178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:40.649908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:40.655636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b5b3c-7d23-4707-a07d-e94dadfc309d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:40.690507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:40.691160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:40.691439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:40.691710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:40.692447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:40.692754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:40.694096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:40.706834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 542/1066] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:40.727086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d30bd0-1488-44a7-8e6f-6cf47bc4b7dd None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:40.766814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:40.767719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:40.768059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:40.768384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:40.780834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:40.789522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:40.790667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:40.796827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 525/1067] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Jun 7 15:46:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:40.797592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:40.909729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16028f-bfa0-42c6-9871-79123a932c70 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:40.945810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:40.946477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:40.946761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:40.947038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:40.947983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:40.948172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:40.949511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:40.958064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 543/1068] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Jun 7 15:46:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:40.963998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.061818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:41.074424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ef1cd-4b21-4886-9011-c12dfe8a4c5e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.108237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.109061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.109345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.109622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.120992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:41.122505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:41.144146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:41.145240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:41.146157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:41.146157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:41.146157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:41.146157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:41.146157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 526/1069] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:41.151485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.171814 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:41.254914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b68611-817a-4078-97d5-38e14e005ca9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.287832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a073471-a73c-4ca9-8d8a-8cf97b4057ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.288525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073471-a73c-4ca9-8d8a-8cf97b4057ec None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.288812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073471-a73c-4ca9-8d8a-8cf97b4057ec None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.289088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073471-a73c-4ca9-8d8a-8cf97b4057ec None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.289804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a073471-a73c-4ca9-8d8a-8cf97b4057ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:41.290109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a073471-a73c-4ca9-8d8a-8cf97b4057ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:41.291473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a073471-a73c-4ca9-8d8a-8cf97b4057ec None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:41.303214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 544/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:41.309480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.421389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b36ce-ab85-47c2-bc4b-86784ba9cc33 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.454701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.455607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.455927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.456207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.464016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:41.465510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:41.468013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:41.470504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba8c30-0184-4e2e-a2ec-a0de8c85ecf1 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.470984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba8c30-0184-4e2e-a2ec-a0de8c85ecf1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.471147 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba8c30-0184-4e2e-a2ec-a0de8c85ecf1 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.472835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 545/1071] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:41.478051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83717537-f0e7-4fc6-aac1-66c0f2e99d15 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.478333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83717537-f0e7-4fc6-aac1-66c0f2e99d15 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.478606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83717537-f0e7-4fc6-aac1-66c0f2e99d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:41.480656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:41.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:41.482601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 527/1072] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:41.487700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.501173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-83717537-f0e7-4fc6-aac1-66c0f2e99d15 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:41.502939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-83717537-f0e7-4fc6-aac1-66c0f2e99d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:41.568518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 546/1073] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:41.587057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40f802-f370-4779-831d-fb34bec6b900 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.591208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83717537-f0e7-4fc6-aac1-66c0f2e99d15 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.601965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:41.621846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.622530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.622816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.623098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.623928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:41.624243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:41.625718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:41.648760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 528/1074] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:41.655125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.702021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83717537-f0e7-4fc6-aac1-66c0f2e99d15 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.714138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:41.735356 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.736146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.737028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.737308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.747209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:41.755578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:41.756765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:41.757805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 547/1075] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:41.760772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71209661-22ce-4e4e-8b6e-65d720cf8251 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.762587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.795229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.796021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.796311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.796639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.807279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:41.808749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:41.826193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:41.827143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:41.828175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:41.828175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:41.828175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:41.828175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:41.828175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 529/1076] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:41.829459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:41.833217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.861707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2f5b-372c-4e21-99fa-8d866dc4b6b4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.895312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.895971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.896251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.896560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.897296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:41.897594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:41.898964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:41.906880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 548/1077] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:41.913200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:41.934790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3ded2-0622-40ef-85eb-fc2e8a42c77a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:41.947158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:41.974148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:41.975320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:41.975320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:41.975558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:41.976597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:41.976721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:41.978104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:41.989663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 530/1078] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:41.995137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:42.023239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79a29-c836-4933-8f73-6b9a6c36c4c9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:42.051755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:42.058067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:42.059036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:42.059321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:42.059594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] PATH_INFO: `/v3/users/a6ecf8652d3e433296aa4e89a8bf803e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:42.060811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] RBAC: Authorizing `identity:get_user(user_id=a6ecf8652d3e433296aa4e89a8bf803e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:42.062228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:42.064004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 549/1079] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/users/a6ecf8652d3e433296aa4e89a8bf803e => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:42.070104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:42.095724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cd8eb-b577-4ada-a59a-7e30b617061e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:42.131754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:42.132582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:42.133060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:42.133169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:42.140034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:42.141500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:42.157462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:42.158604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:42.159524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 531/1080] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:42.164515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:42.172040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a36a0d-d608-448f-947b-d23accacb35c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:42.194096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:42.206695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:42.207548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:42.207826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:42.208100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:42.209284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b70c79e5aef14caaa1b106d33f39f76d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:42.210695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:42.212546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 550/1081] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:46:42 2022] GET /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:42.218214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:42.266851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fd7be-5f6f-48f0-8f6f-de065b870368 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:42.281860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:42.302515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:42.303171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:42.303449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:42.303730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:42.304536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:42.304842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:42.306251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:42.318003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 532/1082] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:42.318162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf413f-b45b-4275-a0d7-ddd541a3b6f7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:42.323842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:42.351083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b75102-dd55-43e8-837e-dce40e8201da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:42.351965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b75102-dd55-43e8-837e-dce40e8201da None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:42.352246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b75102-dd55-43e8-837e-dce40e8201da None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:42.352541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b75102-dd55-43e8-837e-dce40e8201da None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d/users/a6ecf8652d3e433296aa4e89a8bf803e/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:42.355568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b75102-dd55-43e8-837e-dce40e8201da None admin] RBAC: Authorizing `identity:create_grant(project_id=b70c79e5aef14caaa1b106d33f39f76d, user_id=a6ecf8652d3e433296aa4e89a8bf803e, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:42.358322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b75102-dd55-43e8-837e-dce40e8201da None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:42.374585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 551/1083] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:46:42 2022] PUT /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d/users/a6ecf8652d3e433296aa4e89a8bf803e/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:46:42.419430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:42.430534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee82c-636b-4b38-9fd5-2ff651e30b53 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:42.466854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e377aaf-68ac-420c-84e3-ba6e683058c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:42.467754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e377aaf-68ac-420c-84e3-ba6e683058c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:42.468035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e377aaf-68ac-420c-84e3-ba6e683058c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:42.468310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e377aaf-68ac-420c-84e3-ba6e683058c7 None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/0b352e6a51fc4d1f8b9b97d092d11bd7/roles/125bd519889240d29b02cfd21e83bab8` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:42.470614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e377aaf-68ac-420c-84e3-ba6e683058c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=0b352e6a51fc4d1f8b9b97d092d11bd7, role_id=125bd519889240d29b02cfd21e83bab8)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:42.472181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e377aaf-68ac-420c-84e3-ba6e683058c7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:42.487779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 533/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:46:42 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/0b352e6a51fc4d1f8b9b97d092d11bd7/roles/125bd519889240d29b02cfd21e83bab8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:46:44.610827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05e997-e354-46ee-aef1-25a079c4f20b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:44.611366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05e997-e354-46ee-aef1-25a079c4f20b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:44.611366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05e997-e354-46ee-aef1-25a079c4f20b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:44.613022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 552/1085] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:44.618282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811797c-92f2-4ab8-9f58-2d978fae08fa None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:44.618679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811797c-92f2-4ab8-9f58-2d978fae08fa None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:44.618963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811797c-92f2-4ab8-9f58-2d978fae08fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:44.635815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d811797c-92f2-4ab8-9f58-2d978fae08fa None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:44.637032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d811797c-92f2-4ab8-9f58-2d978fae08fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:44.702475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 534/1086] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:44.707556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468a81b-3a74-4148-990e-97d9d8fa23f8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:44.707842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468a81b-3a74-4148-990e-97d9d8fa23f8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:44.708120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468a81b-3a74-4148-990e-97d9d8fa23f8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:44.709622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 553/1087] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:44.714579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b534be1-a572-411d-b4ba-1ed0ee7b4596 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:44.714871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b534be1-a572-411d-b4ba-1ed0ee7b4596 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:44.715153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b534be1-a572-411d-b4ba-1ed0ee7b4596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:44.726682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468a81b-3a74-4148-990e-97d9d8fa23f8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:44.733414 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3b534be1-a572-411d-b4ba-1ed0ee7b4596 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:44.734594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3b534be1-a572-411d-b4ba-1ed0ee7b4596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:44.800802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 535/1088] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:44.823703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b534be1-a572-411d-b4ba-1ed0ee7b4596 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:44.828439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468a81b-3a74-4148-990e-97d9d8fa23f8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:44.834870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:44.855937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:44.867097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:44.867825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:44.868107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:44.868440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:44.878619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:44.886332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:44.887292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:44.888141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 554/1089] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:44.893444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:44.927468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b534be1-a572-411d-b4ba-1ed0ee7b4596 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:44.960428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:44.961233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:44.961514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:44.961789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:44.972093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:46:44.980525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:44.981529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:44.982394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 536/1090] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Jun 7 15:46:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:44.988324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:44.994740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c875ef-7a8c-4527-9150-a90c5c6fc298 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.027636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.028293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.028598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.028872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.029606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:45.030204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.031414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.040040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 555/1091] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:46:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:45.045918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.058239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.089308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c21e4-830c-447f-a964-cf382de6fcf6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.093787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.123536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.124134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.124452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.124733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.125477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:45.125785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.127537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.136042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 537/1092] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Jun 7 15:46:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:45.141492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.150405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446cfc0-f8fa-410d-b2aa-d1f232803c1f None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.186207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.187066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.187344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.187621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] PATH_INFO: `/v3/users/a6ecf8652d3e433296aa4e89a8bf803e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.188815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] RBAC: Authorizing `identity:get_user(user_id=a6ecf8652d3e433296aa4e89a8bf803e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.190617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.192242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 556/1093] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/users/a6ecf8652d3e433296aa4e89a8bf803e => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:45.197793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.246200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c7df4-72e3-4f45-9ac1-426f0c25524e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.274483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.280339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.281146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.281426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.281698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.292215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.293671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.297262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adb78-7f85-4c13-92b2-51b9c9f7ee5b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:45.311275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:45.312302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:45.313398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:45.313398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:45.313398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:45.313398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:45.313398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 538/1094] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:45.318033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.318827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.336952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.337606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.337893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.338175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.410110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.411552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.418696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57a821-f96b-4df7-8f7b-9739076f6d18 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.452766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.453659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.453948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.454220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.454932 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:45.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.456876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.468348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 539/1095] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:45.474996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.510782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 557/1096] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:46:45.516581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.522028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.534454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.576892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ffa3-2e67-4d5b-935b-136b4a659bbd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.612828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.613624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.613920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.614175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.615981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c50da-ca78-4720-903f-3139aee9a9fa None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.621214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.622658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:46:45.637031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:45.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:46:45.639331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 540/1097] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:45.643912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.652509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.653354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.653626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.653892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.655040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b70c79e5aef14caaa1b106d33f39f76d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.656501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.658859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 558/1098] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:45.664588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.743556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3978cd-8715-4ed2-aee0-dd8089071257 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.745920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.748078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:45.763457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4aa2f-675e-4e5f-8bf4-e2607192d8f7 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.778194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.778857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.779139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.779415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.780240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:45.780574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.781992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.797669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.798316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.798598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.798996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.799492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 541/1099] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:45.799855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=b70c79e5aef14caaa1b106d33f39f76d, user.id=a6ecf8652d3e433296aa4e89a8bf803e) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:45.800185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:45.801611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:45.805211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:45.807652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 559/1100] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/role_assignments?user.id=a6ecf8652d3e433296aa4e89a8bf803e&scope.project.id=b70c79e5aef14caaa1b106d33f39f76d&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:45.911399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08afa1-062f-4d49-b61a-33636e741671 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:45.949872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:45.950460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:45.950743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:45.951018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:45.983633 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:46.023655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:46.025142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:46.127693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 542/1101] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:46:46.133489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:46.235332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aaed62-2a3d-4efa-8ea4-33f2a3d6f32c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:46.268991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:46.269780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:46.270060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:46.270331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:46.280585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:46.282014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:46.290299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:46:46.300833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:46.301910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:46.302836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:46:46.302836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:46:46.302836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:46:46.302836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:46.302836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 560/1102] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:46:46.308008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:46.369010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:46.407026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e99d4-4d76-4a68-8a96-92c4e389fd66 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:46.441522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:46.442177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:46.442462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:46.442738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:46.443446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:46.443751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:46.445142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:46.456295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 543/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:46.461842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:46.566963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ae3a4-cb00-4075-a86e-a76e5ca7ff02 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:46.600372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:46.601170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:46.601453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:46.601725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:46.608247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:46.609709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:46:46.624410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:46:46.625841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:46:46.626741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 561/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:46:46.632022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:46.730169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fd559-6668-432c-938a-c0a8a7af2e7e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:46.753677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:46.765537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26f472-afb4-47ad-8762-5e8cf4f44c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:46.766191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f472-afb4-47ad-8762-5e8cf4f44c62 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:46.766473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f472-afb4-47ad-8762-5e8cf4f44c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:46.766743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f472-afb4-47ad-8762-5e8cf4f44c62 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:46.768022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f472-afb4-47ad-8762-5e8cf4f44c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:46.768327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f472-afb4-47ad-8762-5e8cf4f44c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:46.769800 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f472-afb4-47ad-8762-5e8cf4f44c62 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:46.781588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 544/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:46.788071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:46.846148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:46.885386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13d574-057c-4d1c-b59c-f7d50da3d894 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:46.918740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1034ae6-e89e-4075-b9af-f657a7347266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:46.919383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1034ae6-e89e-4075-b9af-f657a7347266 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:46.919663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1034ae6-e89e-4075-b9af-f657a7347266 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:46.919934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1034ae6-e89e-4075-b9af-f657a7347266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:46.920788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1034ae6-e89e-4075-b9af-f657a7347266 None admin] RBAC: Adding query filter params (role.id=125bd519889240d29b02cfd21e83bab8, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0b352e6a51fc4d1f8b9b97d092d11bd7) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:46.921098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1034ae6-e89e-4075-b9af-f657a7347266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:46.922523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1034ae6-e89e-4075-b9af-f657a7347266 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:46.928559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 562/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:46 2022] GET /identity/v3/role_assignments?user.id=0b352e6a51fc4d1f8b9b97d092d11bd7&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=125bd519889240d29b02cfd21e83bab8 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:46:48.127826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fae38-2ceb-4c71-a2dc-c4a3a9f23b73 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:48.128413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fae38-2ceb-4c71-a2dc-c4a3a9f23b73 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:48.128509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fae38-2ceb-4c71-a2dc-c4a3a9f23b73 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:48.129848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 545/1107] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:48.135221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23450a2-ed89-431c-9a60-6b57b64382b7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:48.135656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23450a2-ed89-431c-9a60-6b57b64382b7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:48.135748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23450a2-ed89-431c-9a60-6b57b64382b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:48.153439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c23450a2-ed89-431c-9a60-6b57b64382b7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:48.154598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c23450a2-ed89-431c-9a60-6b57b64382b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:48.218929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 563/1108] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:48.244529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fae38-2ceb-4c71-a2dc-c4a3a9f23b73 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:48.341919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fae38-2ceb-4c71-a2dc-c4a3a9f23b73 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:48.374787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319ea9f-2c26-44c1-8cd3-55bd8dbe89c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:48.375564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319ea9f-2c26-44c1-8cd3-55bd8dbe89c1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:48.375846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319ea9f-2c26-44c1-8cd3-55bd8dbe89c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:48.376118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319ea9f-2c26-44c1-8cd3-55bd8dbe89c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:48.378293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319ea9f-2c26-44c1-8cd3-55bd8dbe89c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:48.379650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319ea9f-2c26-44c1-8cd3-55bd8dbe89c1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:48.381259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 546/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:46:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:48.386818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23450a2-ed89-431c-9a60-6b57b64382b7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:48.438924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:48.481934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23450a2-ed89-431c-9a60-6b57b64382b7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:48.514880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa890-d31d-47e1-9fd3-f0dc951d3188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:48.515533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa890-d31d-47e1-9fd3-f0dc951d3188 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:48.515811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa890-d31d-47e1-9fd3-f0dc951d3188 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:48.516080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa890-d31d-47e1-9fd3-f0dc951d3188 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:48.516806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa890-d31d-47e1-9fd3-f0dc951d3188 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:48.518347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa890-d31d-47e1-9fd3-f0dc951d3188 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:48.537585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 564/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:46:48 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:46:50.816078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208d00-9ec4-4bf6-878e-714b4c0c1089 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:50.816656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208d00-9ec4-4bf6-878e-714b4c0c1089 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:50.816656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208d00-9ec4-4bf6-878e-714b4c0c1089 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:50.818006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 547/1111] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:50.823221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbfb8-7418-48e0-ab5a-10f3a7685f82 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:50.823614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbfb8-7418-48e0-ab5a-10f3a7685f82 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:50.823779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbfb8-7418-48e0-ab5a-10f3a7685f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:50.840698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7c3bbfb8-7418-48e0-ab5a-10f3a7685f82 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:50.841866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7c3bbfb8-7418-48e0-ab5a-10f3a7685f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:50.907082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 565/1112] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:50.930342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27208d00-9ec4-4bf6-878e-714b4c0c1089 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:50.980743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:51.032734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27208d00-9ec4-4bf6-878e-714b4c0c1089 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:51.065892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:51.066784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:51.067007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:51.067277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] PATH_INFO: `/v3/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:51.068740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] RBAC: Authorizing `identity:get_role(role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:51.070091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:51.071443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 548/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:50 2022] GET /identity/v3/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:51.077277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbfb8-7418-48e0-ab5a-10f3a7685f82 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:51.118566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:51.177039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbfb8-7418-48e0-ab5a-10f3a7685f82 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:51.210417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:51.211274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:51.211554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:51.211826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] PATH_INFO: `/v3/users/5d2ca5a543324b3ea0f0748748388ed2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:51.225149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] RBAC: Authorizing `identity:get_user(user_id=5d2ca5a543324b3ea0f0748748388ed2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:51.226566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:51.228199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 566/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:51 2022] GET /identity/v3/users/5d2ca5a543324b3ea0f0748748388ed2 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:51.234044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:51.332148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd752f8-3500-4cb2-b878-58849fc13a87 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:51.350776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:51.368293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:51.368973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:51.369255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:51.369530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:51.434430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:51.435867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:51.530420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 549/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:46:51.537462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:51.630707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:51.634707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9e2c4-209d-4306-a7fa-f682480afe88 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:51.668771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c909e-c0d6-415f-8a59-971eba40230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:51.669617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c909e-c0d6-415f-8a59-971eba40230a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:51.669897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c909e-c0d6-415f-8a59-971eba40230a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:51.670168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c909e-c0d6-415f-8a59-971eba40230a None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:51.671320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c909e-c0d6-415f-8a59-971eba40230a None admin] RBAC: Authorizing `identity:get_project(project_id=b70c79e5aef14caaa1b106d33f39f76d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:51.672773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c909e-c0d6-415f-8a59-971eba40230a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:51.675107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 567/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:46:51 2022] GET /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:51.681291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:51.699231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:51.781251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af9639-856e-4bb9-8e83-6ca1fe4866a3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:51.814587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bf179-3439-44f8-8aef-2855522b74d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:51.815225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bf179-3439-44f8-8aef-2855522b74d6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:51.815513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bf179-3439-44f8-8aef-2855522b74d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:51.815791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bf179-3439-44f8-8aef-2855522b74d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:51.816629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bf179-3439-44f8-8aef-2855522b74d6 None admin] RBAC: Adding query filter params (role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=b70c79e5aef14caaa1b106d33f39f76d, user.id=5d2ca5a543324b3ea0f0748748388ed2) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:51.817147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bf179-3439-44f8-8aef-2855522b74d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:51.818342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bf179-3439-44f8-8aef-2855522b74d6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:51.823628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 550/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:51 2022] GET /identity/v3/role_assignments?user.id=5d2ca5a543324b3ea0f0748748388ed2&scope.project.id=b70c79e5aef14caaa1b106d33f39f76d&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:54.097743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4faba1-30d3-48d5-9478-6d92fb1bcbe0 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:54.098280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4faba1-30d3-48d5-9478-6d92fb1bcbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:54.098280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4faba1-30d3-48d5-9478-6d92fb1bcbe0 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:54.099651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 568/1118] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:54.105041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b10d6-ad11-4480-8a6b-4eeee436dc9a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:54.105477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b10d6-ad11-4480-8a6b-4eeee436dc9a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:54.105594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b10d6-ad11-4480-8a6b-4eeee436dc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:54.122520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-853b10d6-ad11-4480-8a6b-4eeee436dc9a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:54.123678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-853b10d6-ad11-4480-8a6b-4eeee436dc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:54.190050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 551/1119] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:46:54.211832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4faba1-30d3-48d5-9478-6d92fb1bcbe0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:54.309538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4faba1-30d3-48d5-9478-6d92fb1bcbe0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:54.342777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:54.343610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:54.343888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:54.344159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] PATH_INFO: `/v3/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:54.345656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] RBAC: Authorizing `identity:get_role(role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:54.347023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:54.348419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 569/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:54 2022] GET /identity/v3/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:54.354253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b10d6-ad11-4480-8a6b-4eeee436dc9a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:54.368501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:54.453482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b10d6-ad11-4480-8a6b-4eeee436dc9a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:54.485679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:54.487153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:54.487429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:54.487699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] PATH_INFO: `/v3/users/5d2ca5a543324b3ea0f0748748388ed2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:54.488885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=5d2ca5a543324b3ea0f0748748388ed2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:54.490303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:54.491904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 552/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:54 2022] GET /identity/v3/users/5d2ca5a543324b3ea0f0748748388ed2 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:54.497665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:54.530608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:54.593252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e35ec-6e90-455a-994c-e90944839422 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:54.626349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1934c-4b64-43ec-a7bd-475ed5a1a125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:54.627189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1934c-4b64-43ec-a7bd-475ed5a1a125 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:54.627465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1934c-4b64-43ec-a7bd-475ed5a1a125 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:54.627733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1934c-4b64-43ec-a7bd-475ed5a1a125 None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:54.628904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1934c-4b64-43ec-a7bd-475ed5a1a125 None admin] RBAC: Authorizing `identity:get_project(project_id=b70c79e5aef14caaa1b106d33f39f76d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:54.630310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1934c-4b64-43ec-a7bd-475ed5a1a125 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:54.631957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 570/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:46:54 2022] GET /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:54.637766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:54.716194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:54.737811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1cd5-4848-452f-b2e7-d91573b29bd5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:54.770206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89666fa-1231-4a3e-9915-827e532d2721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:54.771079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89666fa-1231-4a3e-9915-827e532d2721 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:54.771352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89666fa-1231-4a3e-9915-827e532d2721 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:54.771624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89666fa-1231-4a3e-9915-827e532d2721 None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d/users/5d2ca5a543324b3ea0f0748748388ed2/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:54.773783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89666fa-1231-4a3e-9915-827e532d2721 None admin] RBAC: Authorizing `identity:create_grant(project_id=b70c79e5aef14caaa1b106d33f39f76d, user_id=5d2ca5a543324b3ea0f0748748388ed2, role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:54.775197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89666fa-1231-4a3e-9915-827e532d2721 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:54.788069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 553/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:46:54 2022] PUT /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d/users/5d2ca5a543324b3ea0f0748748388ed2/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:46:57.046442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f03e51-ff43-4255-9ab9-f9eae22b4612 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:57.046972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f03e51-ff43-4255-9ab9-f9eae22b4612 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:57.046972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f03e51-ff43-4255-9ab9-f9eae22b4612 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:57.049017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 571/1124] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:46:57.054480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8273eb-de73-48e8-9641-cddbc47e98df None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:57.054912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8273eb-de73-48e8-9641-cddbc47e98df None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:57.055066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8273eb-de73-48e8-9641-cddbc47e98df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:57.072096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bf8273eb-de73-48e8-9641-cddbc47e98df None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:46:57.073376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bf8273eb-de73-48e8-9641-cddbc47e98df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:46:57.143716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 554/1125] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:46:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:46:57.167136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f03e51-ff43-4255-9ab9-f9eae22b4612 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:57.264206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f03e51-ff43-4255-9ab9-f9eae22b4612 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:57.265697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:57.298137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:57.298938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:57.299214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:57.299483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] PATH_INFO: `/v3/roles/4d92f154cac84495bb6172c8a120a9b1` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:57.300958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] RBAC: Authorizing `identity:get_role(role_id=4d92f154cac84495bb6172c8a120a9b1)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:57.302301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:57.303629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 572/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:57 2022] GET /identity/v3/roles/4d92f154cac84495bb6172c8a120a9b1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:57.309287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8273eb-de73-48e8-9641-cddbc47e98df None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:57.375722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:57.409184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8273eb-de73-48e8-9641-cddbc47e98df None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:57.443940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:57.444833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:57.445116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:57.445409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] PATH_INFO: `/v3/users/5d2ca5a543324b3ea0f0748748388ed2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:57.446568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] RBAC: Authorizing `identity:get_user(user_id=5d2ca5a543324b3ea0f0748748388ed2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:57.447983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:57.449631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 555/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:46:57 2022] GET /identity/v3/users/5d2ca5a543324b3ea0f0748748388ed2 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:57.455947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:57.552628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0171b09-0956-4e76-ae2a-38df7ddb97e9 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:57.586439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:57.587069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:57.587349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:57.587621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:57.624191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:57.652864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:57.654281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:57.749177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 573/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:46:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:46:57.755635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:57.852775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be606ac2-363a-4f1a-b774-b8d43f893553 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:57.885590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504a883-423c-432f-a3be-689e6b1cf136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:57.886427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504a883-423c-432f-a3be-689e6b1cf136 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:57.886710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504a883-423c-432f-a3be-689e6b1cf136 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:57.886981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504a883-423c-432f-a3be-689e6b1cf136 None admin] PATH_INFO: `/v3/projects/b70c79e5aef14caaa1b106d33f39f76d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:57.888122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504a883-423c-432f-a3be-689e6b1cf136 None admin] RBAC: Authorizing `identity:get_project(project_id=b70c79e5aef14caaa1b106d33f39f76d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:57.889590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504a883-423c-432f-a3be-689e6b1cf136 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:57.891238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 556/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:46:57 2022] GET /identity/v3/projects/b70c79e5aef14caaa1b106d33f39f76d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:46:57.897739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:46:57.963786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:46:57.994151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7da6-c438-4c6a-9ae0-243319f5f654 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:46:58.028059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e43572-f427-4cf1-82d9-dc31f153e047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:46:58.028730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43572-f427-4cf1-82d9-dc31f153e047 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:46:58.029012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43572-f427-4cf1-82d9-dc31f153e047 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:46:58.029285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43572-f427-4cf1-82d9-dc31f153e047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:46:58.030100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e43572-f427-4cf1-82d9-dc31f153e047 None admin] RBAC: Adding query filter params (role.id=4d92f154cac84495bb6172c8a120a9b1, scope.project.id=b70c79e5aef14caaa1b106d33f39f76d, user.id=5d2ca5a543324b3ea0f0748748388ed2) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:46:58.030405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e43572-f427-4cf1-82d9-dc31f153e047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:46:58.031823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e43572-f427-4cf1-82d9-dc31f153e047 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:46:58.038300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 574/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:46:57 2022] GET /identity/v3/role_assignments?user.id=5d2ca5a543324b3ea0f0748748388ed2&scope.project.id=b70c79e5aef14caaa1b106d33f39f76d&role.id=4d92f154cac84495bb6172c8a120a9b1 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:00.276377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811d7b-6c3c-4983-9eac-247c76ae391e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:00.276916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811d7b-6c3c-4983-9eac-247c76ae391e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:00.276916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811d7b-6c3c-4983-9eac-247c76ae391e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:00.278288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 557/1131] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:00.283596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e59675-4d99-4ac2-9c82-5534a7281707 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:00.283993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e59675-4d99-4ac2-9c82-5534a7281707 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:00.284149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e59675-4d99-4ac2-9c82-5534a7281707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:00.301635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-97e59675-4d99-4ac2-9c82-5534a7281707 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:00.302802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-97e59675-4d99-4ac2-9c82-5534a7281707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:00.367432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 575/1132] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:00.391173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811d7b-6c3c-4983-9eac-247c76ae391e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:00.399882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:00.490198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811d7b-6c3c-4983-9eac-247c76ae391e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:00.522922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c209243-0680-4380-8ec3-1064b7f2d7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:00.523699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209243-0680-4380-8ec3-1064b7f2d7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:00.523981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209243-0680-4380-8ec3-1064b7f2d7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:00.524255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209243-0680-4380-8ec3-1064b7f2d7c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:00.525445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c209243-0680-4380-8ec3-1064b7f2d7c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:00.526870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c209243-0680-4380-8ec3-1064b7f2d7c8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:00.528485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 558/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:47:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:00.534920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e59675-4d99-4ac2-9c82-5534a7281707 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:00.593743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:00.630900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e59675-4d99-4ac2-9c82-5534a7281707 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:00.663274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77811849-90fc-43a7-a5b8-f0c2b83c3f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:00.663854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77811849-90fc-43a7-a5b8-f0c2b83c3f83 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:00.664131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77811849-90fc-43a7-a5b8-f0c2b83c3f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:00.664436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77811849-90fc-43a7-a5b8-f0c2b83c3f83 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:00.665131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77811849-90fc-43a7-a5b8-f0c2b83c3f83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:00.666544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77811849-90fc-43a7-a5b8-f0c2b83c3f83 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:00.691057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 576/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:47:00 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:47:02.942531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcb002-80ec-4648-bab5-4fb1ac7ba253 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:02.943092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcb002-80ec-4648-bab5-4fb1ac7ba253 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:02.943092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcb002-80ec-4648-bab5-4fb1ac7ba253 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:02.944477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 559/1135] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:02.949695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf32368-c718-4c36-a356-34ca9ae83246 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:02.950106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf32368-c718-4c36-a356-34ca9ae83246 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:02.950252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf32368-c718-4c36-a356-34ca9ae83246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:02.966800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0bf32368-c718-4c36-a356-34ca9ae83246 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:02.967954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0bf32368-c718-4c36-a356-34ca9ae83246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:03.033867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 577/1136] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:03.058437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcb002-80ec-4648-bab5-4fb1ac7ba253 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:03.133344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:03.160870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcb002-80ec-4648-bab5-4fb1ac7ba253 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:03.194126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555c6ea-0b62-4f5a-9302-faaa35b41498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:03.194873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c6ea-0b62-4f5a-9302-faaa35b41498 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:03.195151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c6ea-0b62-4f5a-9302-faaa35b41498 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:03.195430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c6ea-0b62-4f5a-9302-faaa35b41498 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:03.196602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555c6ea-0b62-4f5a-9302-faaa35b41498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:03.198013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555c6ea-0b62-4f5a-9302-faaa35b41498 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:03.199604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 560/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:47:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:03.205787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf32368-c718-4c36-a356-34ca9ae83246 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:03.250892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:03.303158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf32368-c718-4c36-a356-34ca9ae83246 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:03.335995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef70e4-83d5-4b2e-91b1-62f8aead6d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:03.336679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef70e4-83d5-4b2e-91b1-62f8aead6d26 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:03.336960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef70e4-83d5-4b2e-91b1-62f8aead6d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:03.337234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef70e4-83d5-4b2e-91b1-62f8aead6d26 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:03.337943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef70e4-83d5-4b2e-91b1-62f8aead6d26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:03.339363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef70e4-83d5-4b2e-91b1-62f8aead6d26 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:03.359777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 578/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:47:03 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:47:05.619370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc929-7688-46ee-9f9a-c23d9deb9b6e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:05.619894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc929-7688-46ee-9f9a-c23d9deb9b6e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:05.619894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc929-7688-46ee-9f9a-c23d9deb9b6e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:05.621284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 561/1139] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:05.626573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81885aba-11b5-4772-880c-6ceea74fe9ac None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:05.626998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81885aba-11b5-4772-880c-6ceea74fe9ac None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:05.627129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81885aba-11b5-4772-880c-6ceea74fe9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:05.644225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-81885aba-11b5-4772-880c-6ceea74fe9ac None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:05.645430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-81885aba-11b5-4772-880c-6ceea74fe9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:05.709734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 579/1140] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:05.733533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc929-7688-46ee-9f9a-c23d9deb9b6e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:05.833431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc929-7688-46ee-9f9a-c23d9deb9b6e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:05.866667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:05.867437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:05.867718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:05.867990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:05.878517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:47:05.885947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:47:05.887021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:47:05.888016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 562/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:47:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:47:05.893673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81885aba-11b5-4772-880c-6ceea74fe9ac None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:05.922972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:05.989427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81885aba-11b5-4772-880c-6ceea74fe9ac None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:06.022109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:06.022753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:06.023032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:06.023302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:06.024030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:06.024326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:06.025706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:06.033957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 580/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:47:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:06.039749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:06.042637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:06.138766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b55fb-248c-492c-bfa1-aace29b6e7f1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:06.172753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:06.173861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:06.174143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:06.174420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] PATH_INFO: `/v3/users/0d282d0d5a954c97a61012b32850d1db` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:06.187087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] RBAC: Authorizing `identity:get_user(user_id=0d282d0d5a954c97a61012b32850d1db)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:06.188552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:06.190501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 563/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:47:06 2022] GET /identity/v3/users/0d282d0d5a954c97a61012b32850d1db => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:06.196231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:06.287667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:06.294423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf34cc-afbe-48b7-8bf3-dd11fa2f6d14 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:06.326598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:06.327230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:06.327511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:06.327786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:06.394526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:06.395905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:06.484272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:06.492089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 581/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:47:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:47:06.498101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:06.555040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:06.595176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c183a9-668c-4695-b269-4a9c46c51da5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:06.628084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22a022-be82-4070-9dc7-d89ac3342e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:06.628957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22a022-be82-4070-9dc7-d89ac3342e14 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:06.629234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22a022-be82-4070-9dc7-d89ac3342e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:06.629504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22a022-be82-4070-9dc7-d89ac3342e14 None admin] PATH_INFO: `/v3/projects/a3202cc141db4d0484ec09a495e154ee` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:06.630652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22a022-be82-4070-9dc7-d89ac3342e14 None admin] RBAC: Authorizing `identity:get_project(project_id=a3202cc141db4d0484ec09a495e154ee)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:06.632054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22a022-be82-4070-9dc7-d89ac3342e14 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:06.633739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 564/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:47:06 2022] GET /identity/v3/projects/a3202cc141db4d0484ec09a495e154ee => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:06.639805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:06.734480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa4b70-ba70-4a93-929b-527d0f9e36fc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:06.768684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc28f50-f343-4674-8f9e-b771c921bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:06.769347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28f50-f343-4674-8f9e-b771c921bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:06.769630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28f50-f343-4674-8f9e-b771c921bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:06.769906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28f50-f343-4674-8f9e-b771c921bbc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:06.770733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc28f50-f343-4674-8f9e-b771c921bbc2 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=a3202cc141db4d0484ec09a495e154ee, user.id=0d282d0d5a954c97a61012b32850d1db) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:06.771043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc28f50-f343-4674-8f9e-b771c921bbc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:06.772492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc28f50-f343-4674-8f9e-b771c921bbc2 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:06.778359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 582/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:47:06 2022] GET /identity/v3/role_assignments?user.id=0d282d0d5a954c97a61012b32850d1db&scope.project.id=a3202cc141db4d0484ec09a495e154ee&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:08.968314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98c3c6-3fc2-4d9d-a8a2-14cdd57b4985 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:08.968868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98c3c6-3fc2-4d9d-a8a2-14cdd57b4985 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:08.968868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98c3c6-3fc2-4d9d-a8a2-14cdd57b4985 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:08.970948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 565/1147] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:08.976314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4990-fc86-45cc-8a51-70c1a09c42a1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:08.976760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4990-fc86-45cc-8a51-70c1a09c42a1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:08.976905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f4990-fc86-45cc-8a51-70c1a09c42a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:08.993796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a65f4990-fc86-45cc-8a51-70c1a09c42a1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:08.994960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a65f4990-fc86-45cc-8a51-70c1a09c42a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:09.045542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:09.059408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 583/1148] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:09.081914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98c3c6-3fc2-4d9d-a8a2-14cdd57b4985 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:09.181907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98c3c6-3fc2-4d9d-a8a2-14cdd57b4985 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:09.207434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:09.215206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:09.215989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:09.216269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:09.216571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:09.227315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:47:09.234901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:47:09.236267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:47:09.237257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 566/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:47:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:47:09.242181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f4990-fc86-45cc-8a51-70c1a09c42a1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:09.337385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f4990-fc86-45cc-8a51-70c1a09c42a1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:09.370196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ea490-836f-4606-a861-f57670c0488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:09.370841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ea490-836f-4606-a861-f57670c0488c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:09.371118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ea490-836f-4606-a861-f57670c0488c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:09.371387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ea490-836f-4606-a861-f57670c0488c None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:09.372116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ea490-836f-4606-a861-f57670c0488c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:09.372444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ea490-836f-4606-a861-f57670c0488c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:09.373792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ea490-836f-4606-a861-f57670c0488c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:09.381814 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 584/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:47:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:09.387497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:09.485191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b43f-d6d8-466e-bc01-92a1d49d9996 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:09.518119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:09.518964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:09.519243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:09.519516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] PATH_INFO: `/v3/users/0d282d0d5a954c97a61012b32850d1db` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:09.520700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] RBAC: Authorizing `identity:get_user(user_id=0d282d0d5a954c97a61012b32850d1db)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:09.522126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:09.523722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 567/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:47:09 2022] GET /identity/v3/users/0d282d0d5a954c97a61012b32850d1db => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:09.529400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ea490-836f-4606-a861-f57670c0488c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:09.577111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:09.627264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ea490-836f-4606-a861-f57670c0488c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:09.659409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ecda4-3c64-46c3-bdd0-3a50b45ca540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:09.660242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ecda4-3c64-46c3-bdd0-3a50b45ca540 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:09.660548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ecda4-3c64-46c3-bdd0-3a50b45ca540 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:09.660820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ecda4-3c64-46c3-bdd0-3a50b45ca540 None admin] PATH_INFO: `/v3/projects/a3202cc141db4d0484ec09a495e154ee` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:09.661962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ecda4-3c64-46c3-bdd0-3a50b45ca540 None admin] RBAC: Authorizing `identity:get_project(project_id=a3202cc141db4d0484ec09a495e154ee)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:09.663376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ecda4-3c64-46c3-bdd0-3a50b45ca540 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:09.665052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 585/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:47:09 2022] GET /identity/v3/projects/a3202cc141db4d0484ec09a495e154ee => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:47:09.670858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:09.726289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:09.774905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7690bc-eeee-4d59-b2ea-6c8a32d14756 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:09.816316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422a089-2f0b-4349-9964-5098ba8e5ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:09.817231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422a089-2f0b-4349-9964-5098ba8e5ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:09.817511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422a089-2f0b-4349-9964-5098ba8e5ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:09.817781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422a089-2f0b-4349-9964-5098ba8e5ff8 None admin] PATH_INFO: `/v3/projects/a3202cc141db4d0484ec09a495e154ee/users/0d282d0d5a954c97a61012b32850d1db/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:09.819734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422a089-2f0b-4349-9964-5098ba8e5ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3202cc141db4d0484ec09a495e154ee, user_id=0d282d0d5a954c97a61012b32850d1db, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:09.821186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422a089-2f0b-4349-9964-5098ba8e5ff8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:09.834426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 568/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:47:09 2022] PUT /identity/v3/projects/a3202cc141db4d0484ec09a495e154ee/users/0d282d0d5a954c97a61012b32850d1db/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:47:12.055474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9f33-2688-4b8b-b0b5-0d46ee4caed7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:12.056118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9f33-2688-4b8b-b0b5-0d46ee4caed7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:12.056118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9f33-2688-4b8b-b0b5-0d46ee4caed7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:12.057417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 586/1154] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:12.062815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e6dd5-7dae-470a-a4bb-115f32f920a8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:12.063243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e6dd5-7dae-470a-a4bb-115f32f920a8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:12.063355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e6dd5-7dae-470a-a4bb-115f32f920a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:12.080202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e66e6dd5-7dae-470a-a4bb-115f32f920a8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:12.081407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e66e6dd5-7dae-470a-a4bb-115f32f920a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:12.147949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 569/1155] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:12.172084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c9f33-2688-4b8b-b0b5-0d46ee4caed7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:12.277104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c9f33-2688-4b8b-b0b5-0d46ee4caed7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:12.294083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:12.311069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:12.311782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:12.312062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:12.312333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:12.322405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:47:12.330846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:47:12.332036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:47:12.333175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 587/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:47:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:47:12.338227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e6dd5-7dae-470a-a4bb-115f32f920a8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:12.386299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:12.440922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e6dd5-7dae-470a-a4bb-115f32f920a8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:12.475654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:12.476312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:12.476624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:12.476897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:12.477629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:12.477931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:12.479296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:12.487118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 570/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:47:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:12.492885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:12.595691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933acd2-1fd2-41fd-9e83-017dd56deb78 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:12.628322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:12.629202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:12.629481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:12.629754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] PATH_INFO: `/v3/users/0d282d0d5a954c97a61012b32850d1db` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:12.630924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] RBAC: Authorizing `identity:get_user(user_id=0d282d0d5a954c97a61012b32850d1db)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:12.632347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:12.633976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 588/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:47:12 2022] GET /identity/v3/users/0d282d0d5a954c97a61012b32850d1db => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:12.639831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:12.739546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6f17-0baa-4439-aeba-2e8193136bcc None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:12.748179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:12.773952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:12.774581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:12.774860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:12.775131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:12.846976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:12.848396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:12.948056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 571/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:47:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:47:12.953910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:13.000165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:13.052416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1130d5-dbc1-4e10-95eb-f200c04dd015 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:13.084791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70566fc8-f799-474c-a956-699173942d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:13.085630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70566fc8-f799-474c-a956-699173942d70 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:13.085908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70566fc8-f799-474c-a956-699173942d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:13.086177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70566fc8-f799-474c-a956-699173942d70 None admin] PATH_INFO: `/v3/projects/a3202cc141db4d0484ec09a495e154ee` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:13.087325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70566fc8-f799-474c-a956-699173942d70 None admin] RBAC: Authorizing `identity:get_project(project_id=a3202cc141db4d0484ec09a495e154ee)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:13.088756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70566fc8-f799-474c-a956-699173942d70 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:13.090416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 589/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:47:12 2022] GET /identity/v3/projects/a3202cc141db4d0484ec09a495e154ee => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:13.097692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:13.107958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:13.195891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85084b16-87e9-4654-a1dd-3e67ee683324 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:13.229308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27df76-7c96-4146-91b3-d454614048d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:13.229944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27df76-7c96-4146-91b3-d454614048d7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:13.230223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27df76-7c96-4146-91b3-d454614048d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:13.230493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27df76-7c96-4146-91b3-d454614048d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:13.231291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27df76-7c96-4146-91b3-d454614048d7 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=a3202cc141db4d0484ec09a495e154ee, user.id=0d282d0d5a954c97a61012b32850d1db) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:13.231597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27df76-7c96-4146-91b3-d454614048d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:13.233029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27df76-7c96-4146-91b3-d454614048d7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:13.238725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 572/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:47:13 2022] GET /identity/v3/role_assignments?user.id=0d282d0d5a954c97a61012b32850d1db&scope.project.id=a3202cc141db4d0484ec09a495e154ee&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:15.537673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6567452-1265-4397-81cc-a05870ff2a56 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:15.538183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6567452-1265-4397-81cc-a05870ff2a56 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:15.538183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6567452-1265-4397-81cc-a05870ff2a56 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:15.539500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 590/1162] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:15.544767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d07b-0cdd-4007-a957-734a3485f415 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:15.545177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d07b-0cdd-4007-a957-734a3485f415 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:15.545307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d07b-0cdd-4007-a957-734a3485f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:15.562287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9c08d07b-0cdd-4007-a957-734a3485f415 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:15.563440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9c08d07b-0cdd-4007-a957-734a3485f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:15.627564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 573/1163] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:15.649242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6567452-1265-4397-81cc-a05870ff2a56 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:15.748333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6567452-1265-4397-81cc-a05870ff2a56 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:15.761822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:15.781390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:15.782025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:15.782899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:15.783179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:15.847531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:15.848928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:15.945024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 591/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:47:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:47:15.951002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08d07b-0cdd-4007-a957-734a3485f415 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:15.953201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:16.050083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08d07b-0cdd-4007-a957-734a3485f415 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:16.084948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5152dd5-974f-466e-bbf2-710987433148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:16.086280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5152dd5-974f-466e-bbf2-710987433148 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:16.086650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5152dd5-974f-466e-bbf2-710987433148 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:16.086937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5152dd5-974f-466e-bbf2-710987433148 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:16.094216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5152dd5-974f-466e-bbf2-710987433148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:16.095759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5152dd5-974f-466e-bbf2-710987433148 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f5152dd5-974f-466e-bbf2-710987433148 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:47:16.110113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:47:16.111804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:47:16.113203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 574/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:47:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:47:16.117865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:16.133272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:16.216271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dd5ed-c6db-4c65-890a-8509ca215a30 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:16.249098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719211b9-74d0-4f64-8842-36630b9b15f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:16.249749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719211b9-74d0-4f64-8842-36630b9b15f9 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:16.250027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719211b9-74d0-4f64-8842-36630b9b15f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:16.250301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719211b9-74d0-4f64-8842-36630b9b15f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:16.251655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719211b9-74d0-4f64-8842-36630b9b15f9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:16.251956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719211b9-74d0-4f64-8842-36630b9b15f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:16.253353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719211b9-74d0-4f64-8842-36630b9b15f9 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:16.260250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 592/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Jun 7 15:47:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:18.546137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9171-820b-485e-a5ac-7b69b4c39d91 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:18.546665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9171-820b-485e-a5ac-7b69b4c39d91 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:18.546665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9171-820b-485e-a5ac-7b69b4c39d91 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:18.548592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 575/1167] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:18.553738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab5ec9-a44c-4010-af3b-1dabd9664e09 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:18.554150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab5ec9-a44c-4010-af3b-1dabd9664e09 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:18.554292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab5ec9-a44c-4010-af3b-1dabd9664e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:18.570537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9dab5ec9-a44c-4010-af3b-1dabd9664e09 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:18.571686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9dab5ec9-a44c-4010-af3b-1dabd9664e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:18.635360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 593/1168] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:18.658193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9171-820b-485e-a5ac-7b69b4c39d91 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:18.717844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:18.757115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9171-820b-485e-a5ac-7b69b4c39d91 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:18.788817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920002ab-8b50-4ae9-8db3-cf52dbba06cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:18.789454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920002ab-8b50-4ae9-8db3-cf52dbba06cb None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:18.789735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920002ab-8b50-4ae9-8db3-cf52dbba06cb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:18.790008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920002ab-8b50-4ae9-8db3-cf52dbba06cb None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:18.790730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920002ab-8b50-4ae9-8db3-cf52dbba06cb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:18.792068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920002ab-8b50-4ae9-8db3-cf52dbba06cb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:18.807007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 576/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Tue Jun 7 15:47:18 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:47:21.061654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d151118-53df-4cd1-870e-14e2f98190ba None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:21.062218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d151118-53df-4cd1-870e-14e2f98190ba None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:21.062218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d151118-53df-4cd1-870e-14e2f98190ba None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:21.063866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 594/1170] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:21.069406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a596cf2-5090-43f6-b285-eeb5eba59b7a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:21.069835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a596cf2-5090-43f6-b285-eeb5eba59b7a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:21.069978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a596cf2-5090-43f6-b285-eeb5eba59b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:21.086407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2a596cf2-5090-43f6-b285-eeb5eba59b7a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:21.087588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2a596cf2-5090-43f6-b285-eeb5eba59b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:21.158687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 577/1171] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:21.182174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d151118-53df-4cd1-870e-14e2f98190ba None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:21.193156 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:21.281323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d151118-53df-4cd1-870e-14e2f98190ba None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:21.314914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8a78b-d0a2-47c4-b8ed-c58b7f57b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:21.315703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8a78b-d0a2-47c4-b8ed-c58b7f57b139 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:21.316015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8a78b-d0a2-47c4-b8ed-c58b7f57b139 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:21.316293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8a78b-d0a2-47c4-b8ed-c58b7f57b139 None admin] PATH_INFO: `/v3/domains/596aa597d6cb49ff8c9083503a197b91` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:21.326427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8a78b-d0a2-47c4-b8ed-c58b7f57b139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=596aa597d6cb49ff8c9083503a197b91)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:21.327856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8a78b-d0a2-47c4-b8ed-c58b7f57b139 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:21.330041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 595/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Jun 7 15:47:21 2022] GET /identity/v3/domains/596aa597d6cb49ff8c9083503a197b91 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:21.335852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a596cf2-5090-43f6-b285-eeb5eba59b7a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:21.385912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:21.432615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a596cf2-5090-43f6-b285-eeb5eba59b7a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:21.465465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9aed0f-3beb-4947-b73d-59aa50d1915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:21.466110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aed0f-3beb-4947-b73d-59aa50d1915d None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:21.466394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aed0f-3beb-4947-b73d-59aa50d1915d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:21.466669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aed0f-3beb-4947-b73d-59aa50d1915d None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:21.467370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9aed0f-3beb-4947-b73d-59aa50d1915d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:21.468831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9aed0f-3beb-4947-b73d-59aa50d1915d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:21.493199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 578/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:47:21 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:47:23.757740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7747cfa-825e-46c8-b728-73a88cdda6a4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:23.758268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7747cfa-825e-46c8-b728-73a88cdda6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:23.758382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7747cfa-825e-46c8-b728-73a88cdda6a4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:23.759670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 596/1174] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:23.764925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b4ed8-7809-4b7d-964b-ad901585d8ea None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:23.765360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b4ed8-7809-4b7d-964b-ad901585d8ea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:23.765489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b4ed8-7809-4b7d-964b-ad901585d8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:23.783418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3f3b4ed8-7809-4b7d-964b-ad901585d8ea None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:23.784617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3f3b4ed8-7809-4b7d-964b-ad901585d8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:23.851691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 579/1175] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:23.876730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7747cfa-825e-46c8-b728-73a88cdda6a4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:23.942503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:23.979147 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7747cfa-825e-46c8-b728-73a88cdda6a4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:24.012458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83380e-47c1-45d6-b521-1206f2e0ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:24.013246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83380e-47c1-45d6-b521-1206f2e0ea8c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:24.013520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83380e-47c1-45d6-b521-1206f2e0ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:24.013787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83380e-47c1-45d6-b521-1206f2e0ea8c None admin] PATH_INFO: `/v3/domains/596aa597d6cb49ff8c9083503a197b91` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:24.014937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83380e-47c1-45d6-b521-1206f2e0ea8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=596aa597d6cb49ff8c9083503a197b91)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:24.016435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83380e-47c1-45d6-b521-1206f2e0ea8c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:24.018317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 597/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Jun 7 15:47:23 2022] GET /identity/v3/domains/596aa597d6cb49ff8c9083503a197b91 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:24.023974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b4ed8-7809-4b7d-964b-ad901585d8ea None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:24.057398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:24.121305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b4ed8-7809-4b7d-964b-ad901585d8ea None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:24.159133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990cafa-d6ab-4a7e-8d99-436cd7cfe17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:24.159796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990cafa-d6ab-4a7e-8d99-436cd7cfe17c None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:24.160074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990cafa-d6ab-4a7e-8d99-436cd7cfe17c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:24.160348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990cafa-d6ab-4a7e-8d99-436cd7cfe17c None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:24.161084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990cafa-d6ab-4a7e-8d99-436cd7cfe17c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:24.162505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990cafa-d6ab-4a7e-8d99-436cd7cfe17c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:24.182490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 580/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:47:24 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:47:26.425466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93700450-76bc-4506-9c8f-25b1f67d78d2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:26.425962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93700450-76bc-4506-9c8f-25b1f67d78d2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:26.425962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93700450-76bc-4506-9c8f-25b1f67d78d2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:26.427279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 598/1178] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:26.432454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977b6ad-8b52-4c5b-8337-0005c7b5fbd3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:26.432874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977b6ad-8b52-4c5b-8337-0005c7b5fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:26.433016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977b6ad-8b52-4c5b-8337-0005c7b5fbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:26.450721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7977b6ad-8b52-4c5b-8337-0005c7b5fbd3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:26.451891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7977b6ad-8b52-4c5b-8337-0005c7b5fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:26.516336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 581/1179] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:26.539695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93700450-76bc-4506-9c8f-25b1f67d78d2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:26.638720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93700450-76bc-4506-9c8f-25b1f67d78d2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:26.671917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:26.673426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:26.673799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:26.674073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:26.676667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:26.684566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:47:26.692997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:47:26.694083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:47:26.694997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 599/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:47:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:47:26.700012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977b6ad-8b52-4c5b-8337-0005c7b5fbd3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:26.720914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:26.798775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977b6ad-8b52-4c5b-8337-0005c7b5fbd3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:26.831102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:26.831756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:26.832034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:26.832306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:26.833078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:26.833384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:26.834749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:26.842597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 582/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:47:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:26.849063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:26.948423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4f565-2d90-4828-9560-da6798691a55 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:26.981175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:26.982029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:26.982314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:26.982588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] PATH_INFO: `/v3/users/d386e83de0704ec1a0640e4e4ad42d4f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:26.995617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=d386e83de0704ec1a0640e4e4ad42d4f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:26.997083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:26.998718 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 600/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:47:26 2022] GET /identity/v3/users/d386e83de0704ec1a0640e4e4ad42d4f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:27.005084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:27.084948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:27.103021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9924e-3fd3-4038-a222-d56311c3788e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:27.136027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:27.136706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:27.136988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:27.137263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:27.203726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:27.205584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:27.281902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:27.300918 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 583/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:47:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:47:27.306751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:27.360491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:27.402877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161fde7-ce3e-45fa-ae4b-00c81b2a4ed5 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:27.435054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b51f-ee8f-4e25-bcdb-243e6a925b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:27.435894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b51f-ee8f-4e25-bcdb-243e6a925b73 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:27.436170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b51f-ee8f-4e25-bcdb-243e6a925b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:27.436466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b51f-ee8f-4e25-bcdb-243e6a925b73 None admin] PATH_INFO: `/v3/projects/493eb93510044fefa6b05299428d0bf2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:27.437617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b51f-ee8f-4e25-bcdb-243e6a925b73 None admin] RBAC: Authorizing `identity:get_project(project_id=493eb93510044fefa6b05299428d0bf2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:27.439091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b51f-ee8f-4e25-bcdb-243e6a925b73 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:27.440779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 601/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:47:27 2022] GET /identity/v3/projects/493eb93510044fefa6b05299428d0bf2 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:27.446875 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:27.557950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc04f-9806-4255-af0c-91186b245cd8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:27.591528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdf285-7427-4ffe-ad7b-3a16e71bab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:27.592176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdf285-7427-4ffe-ad7b-3a16e71bab78 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:27.592487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdf285-7427-4ffe-ad7b-3a16e71bab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:27.592764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdf285-7427-4ffe-ad7b-3a16e71bab78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:27.593575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdf285-7427-4ffe-ad7b-3a16e71bab78 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=493eb93510044fefa6b05299428d0bf2, user.id=d386e83de0704ec1a0640e4e4ad42d4f) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:27.593884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdf285-7427-4ffe-ad7b-3a16e71bab78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:27.595335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdf285-7427-4ffe-ad7b-3a16e71bab78 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:27.601098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 584/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:47:27 2022] GET /identity/v3/role_assignments?user.id=d386e83de0704ec1a0640e4e4ad42d4f&scope.project.id=493eb93510044fefa6b05299428d0bf2&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:29.815290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31230cb-e15a-4840-9b23-259c3a259cc9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:29.815846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31230cb-e15a-4840-9b23-259c3a259cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:29.815846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31230cb-e15a-4840-9b23-259c3a259cc9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:29.817253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 602/1186] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:29.822503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c6494-3761-47dd-a2ae-6982b5bd93f6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:29.822937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c6494-3761-47dd-a2ae-6982b5bd93f6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:29.823033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c6494-3761-47dd-a2ae-6982b5bd93f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:29.839860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a84c6494-3761-47dd-a2ae-6982b5bd93f6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:29.841038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a84c6494-3761-47dd-a2ae-6982b5bd93f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:29.882639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:29.906963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 585/1187] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:29.929015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31230cb-e15a-4840-9b23-259c3a259cc9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:30.027605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31230cb-e15a-4840-9b23-259c3a259cc9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:30.049366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:30.060753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:30.061531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:30.061809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:30.062076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:30.072460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:47:30.080640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:47:30.081746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:47:30.082784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 603/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:47:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:47:30.087723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c6494-3761-47dd-a2ae-6982b5bd93f6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:30.191739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c6494-3761-47dd-a2ae-6982b5bd93f6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:30.226794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:30.227458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:30.227742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:30.228021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:30.228788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:30.229096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:30.230477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:30.239162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 586/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:47:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:30.245070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:30.340814 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095ed18-49f7-4a83-aae5-51b8af37364b None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:30.372866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:30.373721 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:30.373996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:30.374265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] PATH_INFO: `/v3/users/d386e83de0704ec1a0640e4e4ad42d4f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:30.375499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] RBAC: Authorizing `identity:get_user(user_id=d386e83de0704ec1a0640e4e4ad42d4f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:30.377701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:30.379302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 604/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:47:30 2022] GET /identity/v3/users/d386e83de0704ec1a0640e4e4ad42d4f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:30.385096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:30.423997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:30.482693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bffa8-2d66-4b05-b045-31ff6dae3d96 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:30.515883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714078d-58ad-4add-a408-57d88645efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:30.516792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714078d-58ad-4add-a408-57d88645efa1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:30.517074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714078d-58ad-4add-a408-57d88645efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:30.517351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714078d-58ad-4add-a408-57d88645efa1 None admin] PATH_INFO: `/v3/projects/493eb93510044fefa6b05299428d0bf2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:30.518539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714078d-58ad-4add-a408-57d88645efa1 None admin] RBAC: Authorizing `identity:get_project(project_id=493eb93510044fefa6b05299428d0bf2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:30.519961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714078d-58ad-4add-a408-57d88645efa1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:30.521667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 587/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:47:30 2022] GET /identity/v3/projects/493eb93510044fefa6b05299428d0bf2 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:30.527520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:30.576633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:30.625860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d82d-0172-4e12-a821-b06da051232c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:30.658214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb805aa0-6e69-44b7-9a37-83afed2f5b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:30.659078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb805aa0-6e69-44b7-9a37-83afed2f5b21 None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:30.659357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb805aa0-6e69-44b7-9a37-83afed2f5b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:30.659630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb805aa0-6e69-44b7-9a37-83afed2f5b21 None admin] PATH_INFO: `/v3/projects/493eb93510044fefa6b05299428d0bf2/users/d386e83de0704ec1a0640e4e4ad42d4f/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:30.661583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb805aa0-6e69-44b7-9a37-83afed2f5b21 None admin] RBAC: Authorizing `identity:create_grant(project_id=493eb93510044fefa6b05299428d0bf2, user_id=d386e83de0704ec1a0640e4e4ad42d4f, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:30.662991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb805aa0-6e69-44b7-9a37-83afed2f5b21 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:30.676155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 605/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:47:30 2022] PUT /identity/v3/projects/493eb93510044fefa6b05299428d0bf2/users/d386e83de0704ec1a0640e4e4ad42d4f/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:47:32.895586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169992c8-d813-41bd-b9a4-91c3fa018949 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:32.896112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169992c8-d813-41bd-b9a4-91c3fa018949 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:32.896112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169992c8-d813-41bd-b9a4-91c3fa018949 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:32.897538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 588/1193] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:47:32.902831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a40ba-772e-4f40-bbe1-31c4cfdfe247 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:32.903252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a40ba-772e-4f40-bbe1-31c4cfdfe247 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:32.903386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a40ba-772e-4f40-bbe1-31c4cfdfe247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:32.920849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9a8a40ba-772e-4f40-bbe1-31c4cfdfe247 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:47:32.922022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9a8a40ba-772e-4f40-bbe1-31c4cfdfe247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:47:32.987444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 606/1194] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:47:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:47:33.010501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169992c8-d813-41bd-b9a4-91c3fa018949 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:33.108907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169992c8-d813-41bd-b9a4-91c3fa018949 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:33.116576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:33.145317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:33.146189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:33.146425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:33.146701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:33.157394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:47:33.164948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:47:33.166026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:47:33.166919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 589/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:47:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:47:33.172019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a40ba-772e-4f40-bbe1-31c4cfdfe247 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:33.212237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:33.270164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a40ba-772e-4f40-bbe1-31c4cfdfe247 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:33.303409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:33.304069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:33.304349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:33.304655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:33.305392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:33.305694 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:33.307046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:33.314883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 607/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:47:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:33.320843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:33.418306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5842e8-7103-49fa-b3db-6d2457e114b9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:33.450113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:33.450968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:33.451249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:33.451524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] PATH_INFO: `/v3/users/d386e83de0704ec1a0640e4e4ad42d4f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:33.452726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] RBAC: Authorizing `identity:get_user(user_id=d386e83de0704ec1a0640e4e4ad42d4f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:33.454819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:33.456447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 590/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Jun 7 15:47:33 2022] GET /identity/v3/users/d386e83de0704ec1a0640e4e4ad42d4f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:33.462149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:33.559008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbd78f-6a31-401a-bba8-70dbb4686615 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:33.560509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:33.593494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:33.594145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:33.594432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:33.594742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:33.659352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:33.660801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:33.755335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 608/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:47:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:47:33.761063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:33.794925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:33.859845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220673a-8d9d-4f34-aec2-cf96dbea8309 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:33.892300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ff029-db8f-4138-85da-5812beb65afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:33.893162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ff029-db8f-4138-85da-5812beb65afd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:33.893444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ff029-db8f-4138-85da-5812beb65afd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:33.893716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ff029-db8f-4138-85da-5812beb65afd None admin] PATH_INFO: `/v3/projects/493eb93510044fefa6b05299428d0bf2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:33.894879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ff029-db8f-4138-85da-5812beb65afd None admin] RBAC: Authorizing `identity:get_project(project_id=493eb93510044fefa6b05299428d0bf2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:33.896302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ff029-db8f-4138-85da-5812beb65afd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:33.897999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 591/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Jun 7 15:47:33 2022] GET /identity/v3/projects/493eb93510044fefa6b05299428d0bf2 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:33.903910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:47:33.906055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:34.001748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96dc2b-b876-46fd-a616-dd0d83efdc19 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:47:34.034770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71fede-c998-4ece-a70c-ee54834db442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:47:34.035353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71fede-c998-4ece-a70c-ee54834db442 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:47:34.035632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71fede-c998-4ece-a70c-ee54834db442 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:47:34.035904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71fede-c998-4ece-a70c-ee54834db442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:47:34.036743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71fede-c998-4ece-a70c-ee54834db442 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=493eb93510044fefa6b05299428d0bf2, user.id=d386e83de0704ec1a0640e4e4ad42d4f) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:47:34.037050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71fede-c998-4ece-a70c-ee54834db442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:47:34.038463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71fede-c998-4ece-a70c-ee54834db442 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:47:34.044076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 609/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:47:33 2022] GET /identity/v3/role_assignments?user.id=d386e83de0704ec1a0640e4e4ad42d4f&scope.project.id=493eb93510044fefa6b05299428d0bf2&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:47:43.887745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=49 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:47:44.041548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=67 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:34.902821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942afa0e-4d7d-4193-9e7d-070aeb1544a4 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:34.903440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942afa0e-4d7d-4193-9e7d-070aeb1544a4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:34.903440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942afa0e-4d7d-4193-9e7d-070aeb1544a4 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:34.904977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 592/1201] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:48:34.910857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e318c-3ecd-4fe9-bdfa-aec8ade15d51 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:34.911432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e318c-3ecd-4fe9-bdfa-aec8ade15d51 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:34.911432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e318c-3ecd-4fe9-bdfa-aec8ade15d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:34.939221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-322e318c-3ecd-4fe9-bdfa-aec8ade15d51 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:48:34.940590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-322e318c-3ecd-4fe9-bdfa-aec8ade15d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:48:35.038286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 610/1202] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:48:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:48:35.068953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942afa0e-4d7d-4193-9e7d-070aeb1544a4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:35.199585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942afa0e-4d7d-4193-9e7d-070aeb1544a4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:35.254542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:35.255468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:35.255751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:35.256118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:35.269603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:35.271252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:35.293001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:35.294205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:35.295245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:35.295245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:35.295245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:35.295245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:35.299513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 593/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:48:35.303205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e318c-3ecd-4fe9-bdfa-aec8ade15d51 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:35.443947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e318c-3ecd-4fe9-bdfa-aec8ade15d51 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:35.484882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:35.493319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355a7e7-0153-4f1f-b09f-811379fc76d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:35.494084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355a7e7-0153-4f1f-b09f-811379fc76d4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:35.494368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355a7e7-0153-4f1f-b09f-811379fc76d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:35.494646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355a7e7-0153-4f1f-b09f-811379fc76d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:35.495373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355a7e7-0153-4f1f-b09f-811379fc76d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:35.495675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355a7e7-0153-4f1f-b09f-811379fc76d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:35.497067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355a7e7-0153-4f1f-b09f-811379fc76d4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:35.511882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 611/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:35.518828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:35.595309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:35.656818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac03396-548f-403a-8d9e-1d50c996e748 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:35.708420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783abf5-86cd-400b-8dd6-27d0baaf1f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:35.708420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783abf5-86cd-400b-8dd6-27d0baaf1f5b None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:35.708420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783abf5-86cd-400b-8dd6-27d0baaf1f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:35.708420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783abf5-86cd-400b-8dd6-27d0baaf1f5b None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:35.708885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783abf5-86cd-400b-8dd6-27d0baaf1f5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:35.708885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783abf5-86cd-400b-8dd6-27d0baaf1f5b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:35.734321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 594/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Jun 7 15:48:35 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:48:38.321686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fd3bb-ff2c-4b7d-987a-a7a1ba4257e2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:38.323368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fd3bb-ff2c-4b7d-987a-a7a1ba4257e2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:38.323663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fd3bb-ff2c-4b7d-987a-a7a1ba4257e2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:38.325281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 612/1206] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:48:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:48:38.332099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0860d9a-7cf3-493d-98ae-96181096d35e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:38.332591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0860d9a-7cf3-493d-98ae-96181096d35e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:38.332701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0860d9a-7cf3-493d-98ae-96181096d35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:38.350781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e0860d9a-7cf3-493d-98ae-96181096d35e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:48:38.355020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e0860d9a-7cf3-493d-98ae-96181096d35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:48:38.428541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 595/1207] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:48:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:48:38.467366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fd3bb-ff2c-4b7d-987a-a7a1ba4257e2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:38.599799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fd3bb-ff2c-4b7d-987a-a7a1ba4257e2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:38.648891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:38.649732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:38.650064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:38.650324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:38.663674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:48:38.672230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:38.673401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:38.674274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 613/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:38.680748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0860d9a-7cf3-493d-98ae-96181096d35e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:38.723889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:38.792203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0860d9a-7cf3-493d-98ae-96181096d35e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:38.833793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:38.834585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:38.834961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:38.835307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:38.836091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:38.836467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:38.838061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:38.849210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 596/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:38.856343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:38.921628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:38.985695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f184e-c035-4fd3-b553-a359575513ba None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:39.029084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:39.029964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:39.030326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:39.030613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:39.043259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:39.044867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:39.065268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:39.066634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:39.069434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:39.069434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:39.069434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:39.069434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:39.071004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 614/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:39.078075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:39.228661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:39.231252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ae441-c58c-4bec-8f21-95e6f0ef1bf0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:39.286229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:39.286823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:39.287099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:39.287371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:39.288084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:39.288422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:39.289824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:39.304053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 597/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:48:39.310907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:39.424936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d7a51-4341-42b1-b912-baca1f86e683 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:39.426892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:39.465056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:39.465861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:39.466132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:39.466532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:39.475319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:39.477051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:48:39.493052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:39.494126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:39.501045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 615/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:48:39.503693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:39.631610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa9b7b-bdd5-41f4-8bcc-5b433e650e0c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:39.679307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:39.680449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:39.680900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:39.681574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:39.682929 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:39.683514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:39.685724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:39.714853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 598/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Jun 7 15:48:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:39.721313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:39.844948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af08849-600b-4a7e-a17f-9e8d8d7ea23a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:39.893989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:39.894960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:39.895449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:39.896013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:39.931586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:39.976620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:39.978087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:40.098258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 616/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:48:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 15:48:40.106064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:40.135726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:40.227357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e3329-8fc7-4498-828c-893c36c0776d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:40.264091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:40.265085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:40.265337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:40.265486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:40.276232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:40.277740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:40.299379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:40.300680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:40.301615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:40.301615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:40.301615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:40.301615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:40.301615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 599/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:48:40.309347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:40.394951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:40.441682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d4f8e-5331-4e39-ad33-c5f9d70e3d45 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:40.480881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:40.481699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:40.482072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:40.482441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:40.483382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:40.484049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:40.485575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:40.500219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 617/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:40.506098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:40.588455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:40.625335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb8079-8a65-40a5-80b1-486ebed0b929 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:40.670296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:40.672063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:40.672442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:40.672779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:40.683406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:40.685067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:48:40.703241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:40.707161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:40.708931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 600/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:40.711126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:40.836775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105648-e773-42cd-a36d-ac3ceb877f18 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:40.871849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:40.887747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb98d6-e16a-4008-9191-d6617d950a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:40.888448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb98d6-e16a-4008-9191-d6617d950a07 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:40.888919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb98d6-e16a-4008-9191-d6617d950a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:40.889092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb98d6-e16a-4008-9191-d6617d950a07 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:40.890543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb98d6-e16a-4008-9191-d6617d950a07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:40.890935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb98d6-e16a-4008-9191-d6617d950a07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:40.892667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb98d6-e16a-4008-9191-d6617d950a07 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:40.908055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 618/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:40.914682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:41.054647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ceb14-fdcf-4971-8df8-a0b81bb5f4f9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:41.093272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:41.101706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d88ae6-0677-4604-a4de-bd7dd6e4ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:41.102447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88ae6-0677-4604-a4de-bd7dd6e4ac4d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:41.102726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88ae6-0677-4604-a4de-bd7dd6e4ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:41.102997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88ae6-0677-4604-a4de-bd7dd6e4ac4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:41.103869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d88ae6-0677-4604-a4de-bd7dd6e4ac4d None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0372966e33f6413ebe529c001bbb2518) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:41.104184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d88ae6-0677-4604-a4de-bd7dd6e4ac4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:41.105710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d88ae6-0677-4604-a4de-bd7dd6e4ac4d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:41.122579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 601/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:48:40 2022] GET /identity/v3/role_assignments?user.id=0372966e33f6413ebe529c001bbb2518&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:43.677137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec7afa-0f85-4d40-95f9-8f74f459b288 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:43.677801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec7afa-0f85-4d40-95f9-8f74f459b288 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:43.677801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec7afa-0f85-4d40-95f9-8f74f459b288 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:43.679258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 619/1220] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:48:43.685095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3bba5-7ade-486b-a45e-c47af9208569 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:43.685638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3bba5-7ade-486b-a45e-c47af9208569 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:43.685638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3bba5-7ade-486b-a45e-c47af9208569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:43.706298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-60b3bba5-7ade-486b-a45e-c47af9208569 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:48:43.707525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-60b3bba5-7ade-486b-a45e-c47af9208569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:48:43.787305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 602/1221] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:48:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:48:43.810703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec7afa-0f85-4d40-95f9-8f74f459b288 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:43.934758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec7afa-0f85-4d40-95f9-8f74f459b288 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:43.972706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:43.973578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:43.973874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:43.974170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:43.985722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:48:43.993787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:43.994817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:43.995716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 620/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:48:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:44.001696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3bba5-7ade-486b-a45e-c47af9208569 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:44.117589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3bba5-7ade-486b-a45e-c47af9208569 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:44.143629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:44.158234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:44.158943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:44.159228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:44.159534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:44.160341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:44.160682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:44.162181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:44.171345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 603/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:48:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:44.177776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:44.234023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:44.302048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a059360-3136-4a89-bd6f-50841a069ca6 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:44.338922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:44.340043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:44.340434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:44.340820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:44.355731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:44.357775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:44.380512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:44.381632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:44.382620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:44.382620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:44.382620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:44.382620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:44.388532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 621/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:44.390626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:44.510324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d5809-1d16-4365-8d2a-19913e7045af None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:44.550451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:44.551126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:44.551412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:44.551689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:44.552433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:44.552745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:44.554111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:44.567884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 604/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:44.573927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:44.681636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:44.689738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2c71b-1c95-48dc-9645-0dbea7f1fedf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:44.728124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:44.728939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:44.729342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:44.729852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:44.741736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:44.743750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:48:44.762261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:44.763427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:44.765607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:44.769222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 622/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:44.770263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:44.819902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:44.882541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999bc50-1569-4fe0-a4f4-55bc319ce67c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:44.923737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:44.924833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:44.925189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:44.925567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:44.926601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:44.926976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:44.928764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:44.947479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 605/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Jun 7 15:48:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:44.954333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:45.076380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b6b35-e1ea-47c8-b377-639c4749b68e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:45.116640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:45.117431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:45.117732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:45.118001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:45.129649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:45.131082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:45.149641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:45.150662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:45.152000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:45.152000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:45.152000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:45.152000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:45.152000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 623/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:45.158096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:45.266075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c4b61-481f-4040-a4ce-95dac0bc18ab None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:45.268471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:45.308430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:45.309168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:45.309455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:45.309750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:45.310660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:45.311071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:45.313109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:45.325761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 606/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:45.331982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:45.341265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:45.440850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ed5e9-c999-4a99-bdbd-12439a257891 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:45.481522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:45.482362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:45.482647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:45.483570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:45.491024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:45.492508 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:48:45.510035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:45.511138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:45.512066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 624/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:48:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 15:48:45.517525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:45.638180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553c7a-49ce-4a31-b712-6e84f1e03cd8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:45.682598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310754a2-8b8d-48dc-b045-586ead8ec243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:45.683275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310754a2-8b8d-48dc-b045-586ead8ec243 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:45.683537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310754a2-8b8d-48dc-b045-586ead8ec243 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:45.683816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310754a2-8b8d-48dc-b045-586ead8ec243 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:45.684887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310754a2-8b8d-48dc-b045-586ead8ec243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:45.685327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310754a2-8b8d-48dc-b045-586ead8ec243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:45.687249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310754a2-8b8d-48dc-b045-586ead8ec243 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:45.707785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 607/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:48:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:45.709099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:45.787833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:45.825969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ecd1-f2bf-49ad-ab47-d1fb3f0d6c88 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:45.862963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff622-1940-4b41-bd36-95e24fe141db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:45.864196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff622-1940-4b41-bd36-95e24fe141db None admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:45.864533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff622-1940-4b41-bd36-95e24fe141db None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:45.864973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff622-1940-4b41-bd36-95e24fe141db None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/0372966e33f6413ebe529c001bbb2518/roles/94bbdda12f68411882dcceaf22124ca4` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:45.888763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bff622-1940-4b41-bd36-95e24fe141db None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=0372966e33f6413ebe529c001bbb2518, role_id=94bbdda12f68411882dcceaf22124ca4)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:45.891642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bff622-1940-4b41-bd36-95e24fe141db None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:45.907888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 625/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:48:45 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/0372966e33f6413ebe529c001bbb2518/roles/94bbdda12f68411882dcceaf22124ca4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:48:48.297902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a00fe-110f-4f7c-bf30-6f92c65e5b7d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:48.298578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a00fe-110f-4f7c-bf30-6f92c65e5b7d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:48.298578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a00fe-110f-4f7c-bf30-6f92c65e5b7d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:48.299814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 608/1233] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:48:48.306011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ab3b-12a8-4bd2-9ed9-2ffae0a3d8b2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:48.306571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ab3b-12a8-4bd2-9ed9-2ffae0a3d8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:48.306725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ab3b-12a8-4bd2-9ed9-2ffae0a3d8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:48.326353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7db6ab3b-12a8-4bd2-9ed9-2ffae0a3d8b2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:48:48.327571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7db6ab3b-12a8-4bd2-9ed9-2ffae0a3d8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:48:48.402202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 626/1234] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:48:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:48:48.432334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a00fe-110f-4f7c-bf30-6f92c65e5b7d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:48.461696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:48.548431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a00fe-110f-4f7c-bf30-6f92c65e5b7d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:48.589255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:48.590080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:48.590372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:48.590651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:48.601095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:48:48.609129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:48.610071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 15:48:48.610926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:48.612138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 609/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Jun 7 15:48:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:48.616790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ab3b-12a8-4bd2-9ed9-2ffae0a3d8b2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:48.670018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:48.730415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ab3b-12a8-4bd2-9ed9-2ffae0a3d8b2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:48.768885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:48.769557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:48.769831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:48.770101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:48.770848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] RBAC: Adding query filter params (name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:48.771211 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:48.772715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:48.781355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 627/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Jun 7 15:48:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:48.787201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:48.890806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:48.909475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177834ee-d827-4fe0-8ee4-5ce98a2e142f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:48.947348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:48.948028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:48.948310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:48.948639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:48.960422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:48.961975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:48.980795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:48.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:48.982757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:48.982757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:48.982757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:48.982757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:48.982757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 610/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:48.988830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:49.099995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffd968-7765-45ee-96e3-7461ba91ed05 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:49.109851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:49.139442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:49.140336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:49.140651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:49.140935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:49.141756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:49.142090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:49.143674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:49.156712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 628/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:49.162416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:49.276218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac43f0-534c-4d07-a36f-49d01b107c26 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:49.311785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:49.312617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:49.312899 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:49.313202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:49.317070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:49.322692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:49.324499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:48:49.341254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:49.343211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:49.344146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 611/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:49.349042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:49.460989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69f4cf-9294-42e9-851a-a435dbdf7daf None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:49.498348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:49.499175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:49.499447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:49.499720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:49.500628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:49.501014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:49.502843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:49.526141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 629/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Jun 7 15:48:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:48:49.532819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:49.646040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d217da-5285-4b63-9428-f264414bdd96 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:49.682987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:49.683650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:49.683929 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:49.684292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:49.768962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:49.778196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:49.780633 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:49.896265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 612/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:48:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:48:49.902443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:49.934772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:50.015965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61639c4-683c-4e87-83c0-730bfd10b152 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:50.050549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:50.051637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:50.051993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:50.052433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:50.066182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:50.068344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:50.089002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:50.090680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:50.092578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:50.092578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:50.092578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:50.092578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:50.092578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 630/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:50.096126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:50.207792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:50.213856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61861-a75e-4b60-bbbe-78c40e4e0420 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:50.254205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:50.254888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:50.255174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:50.255459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:50.256182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:50.256614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:50.258128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:50.272332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 613/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:50.278045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:50.369444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:50.395021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af84b9b-6b00-4a16-8c90-800bed6c6ab4 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:50.435835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:50.437381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:50.437700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:50.438007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:50.446094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:50.447617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:48:50.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:50.465650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:50.466676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 631/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:50.472187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:50.579905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b9848-3b09-451e-8203-b3449f1abdd6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:50.615653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92a15b-5b8a-48b0-90af-c5dbd7cd1c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:50.616430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92a15b-5b8a-48b0-90af-c5dbd7cd1c47 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:50.616824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92a15b-5b8a-48b0-90af-c5dbd7cd1c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:50.616997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92a15b-5b8a-48b0-90af-c5dbd7cd1c47 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:50.617869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92a15b-5b8a-48b0-90af-c5dbd7cd1c47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:50.618228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92a15b-5b8a-48b0-90af-c5dbd7cd1c47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:50.619747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92a15b-5b8a-48b0-90af-c5dbd7cd1c47 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:50.633040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 614/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:50.639919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:50.752595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a15c-d230-4f46-af89-dbe09fd53609 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:50.788435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d3781-7f24-48ab-8b30-08dd2ec2d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:50.789290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d3781-7f24-48ab-8b30-08dd2ec2d8ca None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:50.789684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d3781-7f24-48ab-8b30-08dd2ec2d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:50.790028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d3781-7f24-48ab-8b30-08dd2ec2d8ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:50.790921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d3781-7f24-48ab-8b30-08dd2ec2d8ca None admin] RBAC: Adding query filter params (role.id=94bbdda12f68411882dcceaf22124ca4, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0372966e33f6413ebe529c001bbb2518) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:50.791228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d3781-7f24-48ab-8b30-08dd2ec2d8ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:50.792672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d3781-7f24-48ab-8b30-08dd2ec2d8ca None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:50.799169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 632/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:48:50 2022] GET /identity/v3/role_assignments?user.id=0372966e33f6413ebe529c001bbb2518&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=94bbdda12f68411882dcceaf22124ca4 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:53.403556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035c10-68b9-46fe-92f0-e316fecff058 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:53.404092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035c10-68b9-46fe-92f0-e316fecff058 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:53.404092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035c10-68b9-46fe-92f0-e316fecff058 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:53.406139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 615/1247] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:48:53.412137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91cc8b-292b-4c72-95c1-08f54c3ffec6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:53.412695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91cc8b-292b-4c72-95c1-08f54c3ffec6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:53.412792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91cc8b-292b-4c72-95c1-08f54c3ffec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:53.418168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:53.433791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8f91cc8b-292b-4c72-95c1-08f54c3ffec6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:48:53.435068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8f91cc8b-292b-4c72-95c1-08f54c3ffec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:48:53.514896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 633/1248] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:48:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:48:53.541549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035c10-68b9-46fe-92f0-e316fecff058 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:53.560202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:53.651706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035c10-68b9-46fe-92f0-e316fecff058 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:53.689533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:53.690250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:53.690533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:53.690899 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:53.702541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:48:53.710308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:53.711478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:53.712331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 616/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:53.718684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91cc8b-292b-4c72-95c1-08f54c3ffec6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:53.831504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91cc8b-292b-4c72-95c1-08f54c3ffec6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:53.864617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:53.871026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:53.871880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:53.872195 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:53.872589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:53.873575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:53.873986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:53.875472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:53.885056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 634/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:48:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:53.891374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:53.979431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:54.006212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5e21-fdc4-4116-91a6-ddb259e9e128 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:54.043379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:54.044302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:54.044667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:54.045041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:54.057433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:54.059754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:54.079366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:54.080542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:54.081740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:54.081740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:54.081740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:54.081740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:54.081740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 617/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:54.086936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:54.204297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560fd5d-39ab-409a-8401-f69e1c052f2d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:54.243336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:54.243962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:54.244274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:54.244597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:54.245381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:54.245713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:54.247218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:54.261020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 635/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:54.267426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:54.385515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2da8cd-6a53-40ab-8be6-bc81f3b89081 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:54.414987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:54.423988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:54.424855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:54.425141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:54.425420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:54.434795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:54.436775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:48:54.453084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:54.454311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:54.455459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 618/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:54.462395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:54.523062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:54.571629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfee99-f41f-48d2-bbec-4b307ae0f1dc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:54.609035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:54.609723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:54.610003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:54.610277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:54.611126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:54.611431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:54.612903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:54.635405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 636/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Jun 7 15:48:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:54.642232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:54.753341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352673-41bc-4a81-b564-7f08e8413bcb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:54.789504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:54.790064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:54.790344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:54.790620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:54.852819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:54.868247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:54.870049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:54.982937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 619/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:48:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:48:54.988702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:55.094744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e878e7-ac76-489a-9f20-c3a58b0a9a60 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:55.108452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:55.129842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:55.130644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:55.130926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:55.131202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:55.142032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:55.143469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:55.161376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:55.162413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:55.163343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:55.163343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:55.163343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:55.163343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:55.163343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 637/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:55.168286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:55.254320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:55.266892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a6c0f-0837-4967-8be3-88c99d2db976 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:55.299406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:55.300060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:55.300342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:55.300644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:55.301351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:55.301659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:55.303016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:55.314707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 620/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:55.320173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:55.417241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebd336-1660-46b3-ab39-9a286f30e60a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:55.450068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:55.450836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:55.451112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:55.451442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:55.458463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:55.459873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:48:55.473249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:55.474741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:48:55.476231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 638/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:55.480294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:55.578363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f66bfb-2a9c-4916-a997-55a425847c40 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:55.610710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146049d7-1fc3-45e6-a3ef-77220b2c048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:55.611363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146049d7-1fc3-45e6-a3ef-77220b2c048d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:55.611644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146049d7-1fc3-45e6-a3ef-77220b2c048d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:55.611921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146049d7-1fc3-45e6-a3ef-77220b2c048d None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:55.612715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146049d7-1fc3-45e6-a3ef-77220b2c048d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:55.613023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146049d7-1fc3-45e6-a3ef-77220b2c048d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:55.614441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146049d7-1fc3-45e6-a3ef-77220b2c048d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:55.627246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 621/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:48:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:55.633244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:55.658669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:55.732170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca1ecd-d71c-4977-9cc7-05cc5c25b78c None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:55.766772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b8480-143b-4b19-80cd-e9d534ffcbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:55.767402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b8480-143b-4b19-80cd-e9d534ffcbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:55.767678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b8480-143b-4b19-80cd-e9d534ffcbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:55.767946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b8480-143b-4b19-80cd-e9d534ffcbc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:55.768784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b8480-143b-4b19-80cd-e9d534ffcbc5 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0372966e33f6413ebe529c001bbb2518) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:55.769086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b8480-143b-4b19-80cd-e9d534ffcbc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:55.770494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b8480-143b-4b19-80cd-e9d534ffcbc5 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:55.775913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 639/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:48:55 2022] GET /identity/v3/role_assignments?user.id=0372966e33f6413ebe529c001bbb2518&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:57.982249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885cb3-74dc-4633-86df-41f1e22d4ca1 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:57.983343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885cb3-74dc-4633-86df-41f1e22d4ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:57.983343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885cb3-74dc-4633-86df-41f1e22d4ca1 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:57.984750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 622/1261] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:48:57.989806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a5469-138a-49bc-8703-55cb1082fe77 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:57.990193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a5469-138a-49bc-8703-55cb1082fe77 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:57.990341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a5469-138a-49bc-8703-55cb1082fe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:58.007340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c46a5469-138a-49bc-8703-55cb1082fe77 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:48:58.008537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c46a5469-138a-49bc-8703-55cb1082fe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:48:58.072271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 640/1262] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:48:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:48:58.094552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84885cb3-74dc-4633-86df-41f1e22d4ca1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:58.105509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:58.195366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84885cb3-74dc-4633-86df-41f1e22d4ca1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:58.228248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:58.229058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:58.229339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:58.229610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:58.240409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:48:58.247833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:58.249041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:58.249955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 623/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:58.255147 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a5469-138a-49bc-8703-55cb1082fe77 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:58.273015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:58.352231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a5469-138a-49bc-8703-55cb1082fe77 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:58.385238 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:58.385889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:58.386168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:58.386440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:58.387171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:58.387470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:58.388945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:58.396655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 641/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:48:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:58.402424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:58.466819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:58.501425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396e07f-96d0-45b6-a134-eb35fd311d02 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:58.534171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:58.534951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:58.535237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:58.535516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:58.546454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:58.547893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:58.565025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:58.566045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:58.566970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:58.566970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:58.566970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:58.566970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:58.566970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 624/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:58.571880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:58.651473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:58.671033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91482127-a18f-482a-a0af-5bd3fc193c7a None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:58.703139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:58.703781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:58.704057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:58.704325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:58.705062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:58.705357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:58.706700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:58.718669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 642/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:58.724508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:58.821024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164c5bf-1451-42e0-a4fa-4b62f3e0f42c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:58.840732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:58.855587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:58.856407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:58.856694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:58.856969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:58.865163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:58.866832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:48:58.884712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:58.886137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:58.887520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 625/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:58.893237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:58.997096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4919-002a-4ce9-b012-0e4f1e5f01f1 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:59.030256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:59.030905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:59.031182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:59.031451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:59.032270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:59.032603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:59.034004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:59.042241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:59.050387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 643/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Jun 7 15:48:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:59.055975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:59.153095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f770f-018c-4100-a366-c64c071e31a1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:59.185353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:59.186701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:59.186939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:59.187310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:59.197448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:59.198889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:48:59.215857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:59.216940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:59.217937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:48:59.217937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:48:59.217937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:48:59.217937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:59.217937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 626/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:59.222714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:59.320442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e9ab-14a5-4069-95f2-2f0188a2e2e8 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:59.354444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:59.355084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:59.355360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:59.355629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:59.356332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:59.356659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:59.358850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:59.369579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 644/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:48:59.375651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:59.407097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:59.474270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e989a0a-4bd0-4a6d-a5ec-ec0289219c4e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:59.507603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:59.508407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:59.508691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:59.508964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:59.515755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:59.517210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:48:59.531329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:48:59.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:48:59.533508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 627/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:48:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:48:59.538149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:59.594615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:59.634991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9072a-a463-4cb5-a066-56360c637b4e None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:59.666560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7409f-6c67-42f5-8ea6-f70b0bde3499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:59.667141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7409f-6c67-42f5-8ea6-f70b0bde3499 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:59.667418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7409f-6c67-42f5-8ea6-f70b0bde3499 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:59.667689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7409f-6c67-42f5-8ea6-f70b0bde3499 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:59.668478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7409f-6c67-42f5-8ea6-f70b0bde3499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:48:59.668782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7409f-6c67-42f5-8ea6-f70b0bde3499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:59.670688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7409f-6c67-42f5-8ea6-f70b0bde3499 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:59.682115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 645/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:48:59.688659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:48:59.776111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:48:59.785582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257209-3787-4860-900f-14cc60ef6322 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:48:59.819857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d002541-1e94-411b-a6fc-0f0a6dfb5e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:48:59.820761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002541-1e94-411b-a6fc-0f0a6dfb5e56 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:48:59.821042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002541-1e94-411b-a6fc-0f0a6dfb5e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:48:59.821320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002541-1e94-411b-a6fc-0f0a6dfb5e56 None admin] PATH_INFO: `/v3/projects/29a328fa91894ba5b320490897d0559b/users/0372966e33f6413ebe529c001bbb2518/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:48:59.823233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d002541-1e94-411b-a6fc-0f0a6dfb5e56 None admin] RBAC: Authorizing `identity:create_grant(project_id=29a328fa91894ba5b320490897d0559b, user_id=0372966e33f6413ebe529c001bbb2518, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:48:59.824660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d002541-1e94-411b-a6fc-0f0a6dfb5e56 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:48:59.837769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 628/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Jun 7 15:48:59 2022] PUT /identity/v3/projects/29a328fa91894ba5b320490897d0559b/users/0372966e33f6413ebe529c001bbb2518/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:49:02.089039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f707b-87d0-42c3-a97c-63f617dd48ab None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:02.089600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f707b-87d0-42c3-a97c-63f617dd48ab None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:02.089600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f707b-87d0-42c3-a97c-63f617dd48ab None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:02.090962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 646/1274] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:49:02.097140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb1dd8-b654-4bc7-804f-f51b8d48e90f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:02.097545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb1dd8-b654-4bc7-804f-f51b8d48e90f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:02.097698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb1dd8-b654-4bc7-804f-f51b8d48e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:02.114850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0dfb1dd8-b654-4bc7-804f-f51b8d48e90f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:49:02.116017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0dfb1dd8-b654-4bc7-804f-f51b8d48e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:49:02.181584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 629/1275] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:49:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 15:49:02.204612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f707b-87d0-42c3-a97c-63f617dd48ab None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:02.302856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f707b-87d0-42c3-a97c-63f617dd48ab None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:02.319375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:02.335824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:02.336625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:02.336902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:02.337170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:02.347649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 15:49:02.355477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:02.356570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:02.358317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 647/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Jun 7 15:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:02.363023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb1dd8-b654-4bc7-804f-f51b8d48e90f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:02.442071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:02.460682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb1dd8-b654-4bc7-804f-f51b8d48e90f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:02.493983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:02.494632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:02.494911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:02.495182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:02.495907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:02.496206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:02.497590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:02.505657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 630/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Jun 7 15:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:02.511146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:02.608879 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f11e-fe0d-484b-afca-e1adc1ae3485 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:02.641833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:02.642590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:02.642930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:02.643203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:02.653305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:02.655092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:49:02.672262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:02.673386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:02.674342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:49:02.674342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:49:02.674342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:49:02.674342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:02.674342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 648/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:02.679184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:02.778494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d11441-34b6-483a-ad09-3d7921192645 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:02.811610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:02.812254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:02.812559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:02.812829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:02.813526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:02.813826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:02.815172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:02.826479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 631/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:02.832189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:02.864198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:02.943230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dc3a1-2d64-4f7e-b68a-1ad8e8f5744d None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:02.976219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:02.977048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:02.977328 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:02.977597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:02.985599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:02.987023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 15:49:03.001130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:03.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:03.003856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 649/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:49:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:03.008129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:03.011893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:03.105957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a066-37a3-423d-849d-15a0b56296c2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:03.139704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:03.140402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:03.140688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:03.140963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:03.141785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:03.142094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:03.143534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:03.161025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 632/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Jun 7 15:49:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:03.166696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:03.254851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:03.264847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ebfea-0464-4337-b19a-75175d7e66bc None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:03.297431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:03.298057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:03.298336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:03.298606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:03.363317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:03.364778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:03.448872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:03.459610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 650/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Jun 7 15:49:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 15:49:03.465329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:03.525651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:03.562987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4784-8fcc-4eae-933a-0b76e607f551 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:03.595584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:03.596383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:03.596672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:03.596945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:03.606943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:03.608460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 15:49:03.626047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:03.627054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:03.627979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 15:49:03.627979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 15:49:03.627979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 15:49:03.627979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:49:03.627979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 633/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:03.632798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:03.728762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379961-b5ad-409f-ae23-fcf466d34057 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:03.761275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:03.761922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:03.762201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:03.762472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:03.763177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:03.763479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:03.765046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:03.775841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 651/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Jun 7 15:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:03.781935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:03.882805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0266ea2-1d93-42bd-9964-fc921af2eb2d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:03.897589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:03.917962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:03.918747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:03.919025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:03.919292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:03.926613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:03.928049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 15:49:03.942137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:03.943194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jun 07 15:49:03.944308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 634/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Jun 7 15:49:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:03.949186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:03.998769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:04.047442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a502-503d-4801-b83f-5cba36eb1d11 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:04.079402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e930654-f6ce-4d82-857b-341bb79c92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:04.080040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e930654-f6ce-4d82-857b-341bb79c92b0 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:04.080317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e930654-f6ce-4d82-857b-341bb79c92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:04.080610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e930654-f6ce-4d82-857b-341bb79c92b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:04.081373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e930654-f6ce-4d82-857b-341bb79c92b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:04.081673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e930654-f6ce-4d82-857b-341bb79c92b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:04.083070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e930654-f6ce-4d82-857b-341bb79c92b0 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:04.095003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 652/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Jun 7 15:49:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:04.101254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:04.208175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0becbf7-c16c-40dc-b59d-7e18db11fe59 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:04.241439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f7ab6-4bdc-4788-b30e-ed83f7cc01e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:04.242085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f7ab6-4bdc-4788-b30e-ed83f7cc01e5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:04.242366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f7ab6-4bdc-4788-b30e-ed83f7cc01e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:04.242637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f7ab6-4bdc-4788-b30e-ed83f7cc01e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:04.243448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f7ab6-4bdc-4788-b30e-ed83f7cc01e5 None admin] RBAC: Adding query filter params (role.id=5b74a32ffd254d2fbe8013955251c61e, scope.project.id=29a328fa91894ba5b320490897d0559b, user.id=0372966e33f6413ebe529c001bbb2518) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:04.243757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f7ab6-4bdc-4788-b30e-ed83f7cc01e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:04.245228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f7ab6-4bdc-4788-b30e-ed83f7cc01e5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:04.251609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 635/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Jun 7 15:49:04 2022] GET /identity/v3/role_assignments?user.id=0372966e33f6413ebe529c001bbb2518&scope.project.id=29a328fa91894ba5b320490897d0559b&role.id=5b74a32ffd254d2fbe8013955251c61e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:06.480249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72949f-9574-4b33-9a81-82783ff2e052 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:06.480877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72949f-9574-4b33-9a81-82783ff2e052 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:06.480877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72949f-9574-4b33-9a81-82783ff2e052 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:06.482108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 653/1288] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:49:06.487389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe77115-725b-40a9-ae9f-671eb1f9eba9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:06.487785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe77115-725b-40a9-ae9f-671eb1f9eba9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:06.487948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe77115-725b-40a9-ae9f-671eb1f9eba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:06.505170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-afe77115-725b-40a9-ae9f-671eb1f9eba9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:49:06.506345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-afe77115-725b-40a9-ae9f-671eb1f9eba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:49:06.535159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:06.571973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 636/1289] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:49:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:49:06.594844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72949f-9574-4b33-9a81-82783ff2e052 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:06.693848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72949f-9574-4b33-9a81-82783ff2e052 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:06.705895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:06.729312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:06.730090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:06.730413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:06.730631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:06.731262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:06.732759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:06.744434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:06.745637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:06.746756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 654/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:49:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:06.752380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe77115-725b-40a9-ae9f-671eb1f9eba9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:06.850021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe77115-725b-40a9-ae9f-671eb1f9eba9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:06.886793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657dc45-ebc7-4ef3-97fe-153698962c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:06.887434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657dc45-ebc7-4ef3-97fe-153698962c9a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:06.887714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657dc45-ebc7-4ef3-97fe-153698962c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:06.887990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657dc45-ebc7-4ef3-97fe-153698962c9a None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:06.888996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657dc45-ebc7-4ef3-97fe-153698962c9a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:06.889175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657dc45-ebc7-4ef3-97fe-153698962c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:06.890460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657dc45-ebc7-4ef3-97fe-153698962c9a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:06.895550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 637/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Jun 7 15:49:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:06.901516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:07.130786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e05029-fe4f-43d7-8bde-bb5a6a2d0529 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:07.164636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a658b-1559-495b-81a4-c24a9f9f2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:07.165286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a658b-1559-495b-81a4-c24a9f9f2b11 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:07.165562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a658b-1559-495b-81a4-c24a9f9f2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:07.165830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a658b-1559-495b-81a4-c24a9f9f2b11 None admin] PATH_INFO: `/v3/services` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:07.166514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a658b-1559-495b-81a4-c24a9f9f2b11 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:07.166810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a658b-1559-495b-81a4-c24a9f9f2b11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:07.168142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a658b-1559-495b-81a4-c24a9f9f2b11 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:07.172645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 655/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Jun 7 15:49:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:49:09.409753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd16df-d634-4d45-9ce8-0ef3674d7947 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:09.410254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd16df-d634-4d45-9ce8-0ef3674d7947 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:09.410254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd16df-d634-4d45-9ce8-0ef3674d7947 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:09.411558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 638/1293] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:49:09.416789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d1ce5-4e9c-4622-90ac-50c9929d59df None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:09.417229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d1ce5-4e9c-4622-90ac-50c9929d59df None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:09.417332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d1ce5-4e9c-4622-90ac-50c9929d59df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:09.434006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4c5d1ce5-4e9c-4622-90ac-50c9929d59df None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:49:09.435188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4c5d1ce5-4e9c-4622-90ac-50c9929d59df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:49:09.463906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:09.501063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 656/1294] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:49:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:49:09.523690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd16df-d634-4d45-9ce8-0ef3674d7947 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:09.553958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:09.622132 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd16df-d634-4d45-9ce8-0ef3674d7947 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:09.655383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ba7d4-0964-4cb7-a935-824c74ee653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:09.656028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba7d4-0964-4cb7-a935-824c74ee653e None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:09.656308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba7d4-0964-4cb7-a935-824c74ee653e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:09.656607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba7d4-0964-4cb7-a935-824c74ee653e None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:09.657309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ba7d4-0964-4cb7-a935-824c74ee653e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:09.658657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ba7d4-0964-4cb7-a935-824c74ee653e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:09.670749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 639/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Jun 7 15:49:09 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:49:11.935050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b416d-d207-46c3-ae8f-da1585585d4c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:11.936182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b416d-d207-46c3-ae8f-da1585585d4c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:11.936182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b416d-d207-46c3-ae8f-da1585585d4c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:11.937542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 657/1296] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:49:11.942715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718fbfb-fe46-447b-8616-515f30e89f22 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:11.943104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718fbfb-fe46-447b-8616-515f30e89f22 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:11.943269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718fbfb-fe46-447b-8616-515f30e89f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:11.961065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e718fbfb-fe46-447b-8616-515f30e89f22 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:49:11.962262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e718fbfb-fe46-447b-8616-515f30e89f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:49:12.039491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 640/1297] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:49:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:49:12.063483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b416d-d207-46c3-ae8f-da1585585d4c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:12.162309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b416d-d207-46c3-ae8f-da1585585d4c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:12.195863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:12.196615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:12.196897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:12.197172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:12.197813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:12.199159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:12.210931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:12.212090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:12.213159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 658/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:49:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:12.218509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718fbfb-fe46-447b-8616-515f30e89f22 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:12.237286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:12.321677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718fbfb-fe46-447b-8616-515f30e89f22 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:12.354202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:12.354856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:12.355137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:12.355412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:12.356106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:12.356438 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:12.358323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:12.363312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 641/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Jun 7 15:49:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:12.369288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:12.400569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:12.468108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd199e-affc-4ca4-a2ae-4b3b01b2c5e3 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:12.502212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bebf07-7e38-45c5-bf73-263499bea796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:12.502856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bebf07-7e38-45c5-bf73-263499bea796 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:12.503143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bebf07-7e38-45c5-bf73-263499bea796 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:12.503421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bebf07-7e38-45c5-bf73-263499bea796 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:12.504202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bebf07-7e38-45c5-bf73-263499bea796 None admin] RBAC: Adding query filter params (interface=public, service_id=b2226e781a9e4bf29b0bcc19446554a2, region_id=RegionOne) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:12.504541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bebf07-7e38-45c5-bf73-263499bea796 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:12.505900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bebf07-7e38-45c5-bf73-263499bea796 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:12.510804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 659/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Jun 7 15:49:12 2022] GET /identity/v3/endpoints?service_id=b2226e781a9e4bf29b0bcc19446554a2&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:12.516683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:12.595594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:12.613897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a35dc-e019-49d3-bfa0-333110e599b5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:12.645882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daa83d-e198-4c0d-8a2d-ca7270699163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:12.646531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa83d-e198-4c0d-8a2d-ca7270699163 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:12.646809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa83d-e198-4c0d-8a2d-ca7270699163 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:12.647080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa83d-e198-4c0d-8a2d-ca7270699163 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:12.647740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daa83d-e198-4c0d-8a2d-ca7270699163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:12.649106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daa83d-e198-4c0d-8a2d-ca7270699163 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:12.656238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 642/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Jun 7 15:49:12 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:49:14.866449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753112b-f742-4cdc-a7cc-3bd83e597476 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:14.867085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753112b-f742-4cdc-a7cc-3bd83e597476 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:14.867085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753112b-f742-4cdc-a7cc-3bd83e597476 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:14.868375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 660/1302] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:49:14.874101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ac2c-84bf-4147-808e-68f6191e11cf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:14.874503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ac2c-84bf-4147-808e-68f6191e11cf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:14.874661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ac2c-84bf-4147-808e-68f6191e11cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:14.891728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e438ac2c-84bf-4147-808e-68f6191e11cf None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:49:14.892938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e438ac2c-84bf-4147-808e-68f6191e11cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:49:14.958383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 643/1303] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:49:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 15:49:14.981462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753112b-f742-4cdc-a7cc-3bd83e597476 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:15.071710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:15.079270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753112b-f742-4cdc-a7cc-3bd83e597476 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:15.112165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:15.112971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:15.113252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:15.113523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:15.114162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:15.115506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 15:49:15.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 15:49:15.129046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application with Lock( Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self._enter() Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application return self.creator() Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application created_value = creator( Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: ERROR keystone.server.flask.application Jun 07 15:49:15.130269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 661/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Jun 7 15:49:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 15:49:15.135091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438ac2c-84bf-4147-808e-68f6191e11cf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:15.210068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:15.233230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438ac2c-84bf-4147-808e-68f6191e11cf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:15.266152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c1198-0af1-4514-843e-eba66b16ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:15.266806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c1198-0af1-4514-843e-eba66b16ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:15.267090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c1198-0af1-4514-843e-eba66b16ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:15.267366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c1198-0af1-4514-843e-eba66b16ffa3 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:15.268062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c1198-0af1-4514-843e-eba66b16ffa3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:49:15.268383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c1198-0af1-4514-843e-eba66b16ffa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:15.269760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c1198-0af1-4514-843e-eba66b16ffa3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:15.274761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 644/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Jun 7 15:49:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:49:15.280716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:49:15.390848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad593f-8282-4a50-bbe6-93ebb8243667 None admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:49:15.423701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c94c90-b466-41b7-9f62-028b90c70292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:49:15.424335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c94c90-b466-41b7-9f62-028b90c70292 None admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:49:15.424640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c94c90-b466-41b7-9f62-028b90c70292 None admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:49:15.424910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c94c90-b466-41b7-9f62-028b90c70292 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:49:15.425647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c94c90-b466-41b7-9f62-028b90c70292 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:49:15.427159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c94c90-b466-41b7-9f62-028b90c70292 None admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:49:15.445501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 662/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Jun 7 15:49:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:49:25.272191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=30 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:49:25.441467 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:19.755316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f208d-e5d6-4449-8f39-387fed1234d8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:19.755884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f208d-e5d6-4449-8f39-387fed1234d8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:19.755884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f208d-e5d6-4449-8f39-387fed1234d8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:19.757259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 645/1307] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:19.763011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a057d-4582-47d1-9e14-9c495f5e7ab7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:19.763447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a057d-4582-47d1-9e14-9c495f5e7ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:19.763657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a057d-4582-47d1-9e14-9c495f5e7ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:19.842326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-041a057d-4582-47d1-9e14-9c495f5e7ab7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:19.847184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-041a057d-4582-47d1-9e14-9c495f5e7ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:20.091587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 663/1308] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:50:20.142713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c925f-fac6-456a-a6ea-8a049886ac4e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:20.143078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c925f-fac6-456a-a6ea-8a049886ac4e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:20.143431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c925f-fac6-456a-a6ea-8a049886ac4e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:20.145116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 646/1309] 10.222.0.18 () {54 vars in 964 bytes} [Tue Jun 7 15:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:20.151738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2590c3-d412-468e-ac28-13d8a858c0b4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:20.152578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2590c3-d412-468e-ac28-13d8a858c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:20.152857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2590c3-d412-468e-ac28-13d8a858c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:20.233148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2c2590c3-d412-468e-ac28-13d8a858c0b4 None None] MFA Rules not processed for user `dba9f1fa257a4f44ae159850f9ba349c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:20.234483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2c2590c3-d412-468e-ac28-13d8a858c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:20.256585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:20.318463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 664/1310] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Jun 7 15:50:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:20.332493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c925f-fac6-456a-a6ea-8a049886ac4e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:20.334381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a15c925f-fac6-456a-a6ea-8a049886ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:20.471225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c925f-fac6-456a-a6ea-8a049886ac4e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:20.552197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76bb1e-b424-474a-9809-9e643a431a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:20.552895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76bb1e-b424-474a-9809-9e643a431a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:20.553176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76bb1e-b424-474a-9809-9e643a431a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:20.553446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76bb1e-b424-474a-9809-9e643a431a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:20.555089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ba76bb1e-b424-474a-9809-9e643a431a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:20.622522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:20.688419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76bb1e-b424-474a-9809-9e643a431a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:20.689914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76bb1e-b424-474a-9809-9e643a431a98 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:20.712878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 647/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:23.311791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b633-84b8-4d11-a97d-0f3f2649fd5f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:23.312307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b633-84b8-4d11-a97d-0f3f2649fd5f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:23.312307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b633-84b8-4d11-a97d-0f3f2649fd5f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:23.313857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 665/1312] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:23.318926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabf6e5-335c-4f73-8ea6-046317f64656 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:23.319374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabf6e5-335c-4f73-8ea6-046317f64656 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:23.319480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabf6e5-335c-4f73-8ea6-046317f64656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:23.408182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-baabf6e5-335c-4f73-8ea6-046317f64656 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:23.409582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-baabf6e5-335c-4f73-8ea6-046317f64656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:23.515387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 648/1313] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:23.553626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36b633-84b8-4d11-a97d-0f3f2649fd5f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:23.555250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ad36b633-84b8-4d11-a97d-0f3f2649fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:23.665119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36b633-84b8-4d11-a97d-0f3f2649fd5f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:23.744135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:23.747067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7438b-360c-4f5f-b79b-87363b8445f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:23.747853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7438b-360c-4f5f-b79b-87363b8445f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:23.748531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7438b-360c-4f5f-b79b-87363b8445f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:23.749141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7438b-360c-4f5f-b79b-87363b8445f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:23.751150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7ea7438b-360c-4f5f-b79b-87363b8445f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:23.873253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7438b-360c-4f5f-b79b-87363b8445f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:23.874868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7438b-360c-4f5f-b79b-87363b8445f3 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:23.894745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 666/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:50:26.528637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee34bc-46f6-4bf7-8222-9f8b23e860ad None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:26.529205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee34bc-46f6-4bf7-8222-9f8b23e860ad None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:26.529205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee34bc-46f6-4bf7-8222-9f8b23e860ad None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:26.530531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 649/1315] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:26.535719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6a46d-cb9d-47ff-9330-566ef8a0ab00 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:26.536162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6a46d-cb9d-47ff-9330-566ef8a0ab00 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:26.536283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6a46d-cb9d-47ff-9330-566ef8a0ab00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:26.600253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-33e6a46d-cb9d-47ff-9330-566ef8a0ab00 None None] MFA Rules not processed for user `841797717d6e4222a5d0793b3aeb3235`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:26.601724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-33e6a46d-cb9d-47ff-9330-566ef8a0ab00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:26.652639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:26.696184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 667/1316] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:26.736479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee34bc-46f6-4bf7-8222-9f8b23e860ad None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:26.738032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-19ee34bc-46f6-4bf7-8222-9f8b23e860ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:26.748511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:26.840751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee34bc-46f6-4bf7-8222-9f8b23e860ad None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:26.913481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03144e2f-b53e-47d8-bc21-1ddaa1db297c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:26.914121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03144e2f-b53e-47d8-bc21-1ddaa1db297c service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:26.914402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03144e2f-b53e-47d8-bc21-1ddaa1db297c service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:26.914674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03144e2f-b53e-47d8-bc21-1ddaa1db297c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:26.916299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-03144e2f-b53e-47d8-bc21-1ddaa1db297c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:26.978769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:27.025671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03144e2f-b53e-47d8-bc21-1ddaa1db297c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:27.027268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03144e2f-b53e-47d8-bc21-1ddaa1db297c service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:27.045388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 650/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:29.849348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337ef3-297e-448f-9295-704a697723db None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:29.849888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337ef3-297e-448f-9295-704a697723db None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:29.849888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337ef3-297e-448f-9295-704a697723db None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:29.851520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 668/1318] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:29.857407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be06f-4ece-4f73-bbe3-b8c46c2c26d4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:29.857827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be06f-4ece-4f73-bbe3-b8c46c2c26d4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:29.857954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be06f-4ece-4f73-bbe3-b8c46c2c26d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:29.924475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e36be06f-4ece-4f73-bbe3-b8c46c2c26d4 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:29.925726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e36be06f-4ece-4f73-bbe3-b8c46c2c26d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:30.022656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 651/1319] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:30.061713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad337ef3-297e-448f-9295-704a697723db None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:30.063251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ad337ef3-297e-448f-9295-704a697723db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:30.165566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad337ef3-297e-448f-9295-704a697723db None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:30.237103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9dc7d-4218-43b7-af2c-1a6224242d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:30.237739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9dc7d-4218-43b7-af2c-1a6224242d12 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:30.238014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9dc7d-4218-43b7-af2c-1a6224242d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:30.238284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9dc7d-4218-43b7-af2c-1a6224242d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:30.239920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-32a9dc7d-4218-43b7-af2c-1a6224242d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:30.247334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:30.346790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9dc7d-4218-43b7-af2c-1a6224242d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:30.348235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9dc7d-4218-43b7-af2c-1a6224242d12 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:30.366460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 669/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:33.269291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afe465-da89-4638-8c24-b3477f886fee None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:33.269853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afe465-da89-4638-8c24-b3477f886fee None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:33.269853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afe465-da89-4638-8c24-b3477f886fee None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:33.271336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 652/1321] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:33.276331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417512a-8bf5-408d-be02-132311bca070 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:33.276755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417512a-8bf5-408d-be02-132311bca070 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:33.277044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417512a-8bf5-408d-be02-132311bca070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:33.341997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4417512a-8bf5-408d-be02-132311bca070 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:33.343198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4417512a-8bf5-408d-be02-132311bca070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:33.400053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:33.439207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 670/1322] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:33.476582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afe465-da89-4638-8c24-b3477f886fee None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:33.478113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-89afe465-da89-4638-8c24-b3477f886fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:33.497770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:33.583363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afe465-da89-4638-8c24-b3477f886fee None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:33.658586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc1d2d-f663-4023-82e3-51ba400860dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:33.659182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc1d2d-f663-4023-82e3-51ba400860dd service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:33.659460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc1d2d-f663-4023-82e3-51ba400860dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:33.659731 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc1d2d-f663-4023-82e3-51ba400860dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:33.661383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3dbc1d2d-f663-4023-82e3-51ba400860dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:33.736550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:33.777422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc1d2d-f663-4023-82e3-51ba400860dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:33.779282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc1d2d-f663-4023-82e3-51ba400860dd service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:33.797008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 653/1323] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:36.100202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efcd50-45ba-4ea7-b898-0d1fd74a19d2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:36.100811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efcd50-45ba-4ea7-b898-0d1fd74a19d2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:36.100811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efcd50-45ba-4ea7-b898-0d1fd74a19d2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:36.102575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 671/1324] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:36.108063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a2f3c-e07e-4793-a9b7-ac6973a0cfac None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:36.108509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a2f3c-e07e-4793-a9b7-ac6973a0cfac None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:36.108659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a2f3c-e07e-4793-a9b7-ac6973a0cfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:36.175229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ed6a2f3c-e07e-4793-a9b7-ac6973a0cfac None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:36.176516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed6a2f3c-e07e-4793-a9b7-ac6973a0cfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:36.275807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 654/1325] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:36.312713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efcd50-45ba-4ea7-b898-0d1fd74a19d2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:36.314319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-99efcd50-45ba-4ea7-b898-0d1fd74a19d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:36.416895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efcd50-45ba-4ea7-b898-0d1fd74a19d2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:36.490871 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e442e-89d1-4124-b4e3-0ab8630691f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:36.491516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e442e-89d1-4124-b4e3-0ab8630691f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:36.491793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e442e-89d1-4124-b4e3-0ab8630691f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:36.492066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e442e-89d1-4124-b4e3-0ab8630691f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:36.493715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-591e442e-89d1-4124-b4e3-0ab8630691f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:36.511388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:36.605418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e442e-89d1-4124-b4e3-0ab8630691f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:36.607686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e442e-89d1-4124-b4e3-0ab8630691f3 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:36.625413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 672/1326] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:38.928907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2116d-8d92-41cd-a6ac-f75ec6a3e3d0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:38.929478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2116d-8d92-41cd-a6ac-f75ec6a3e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:38.929478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2116d-8d92-41cd-a6ac-f75ec6a3e3d0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:38.930958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 655/1327] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:38.936134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0ca02-92ef-4608-bf1d-d6162c7c1866 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:38.936598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0ca02-92ef-4608-bf1d-d6162c7c1866 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:38.936813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0ca02-92ef-4608-bf1d-d6162c7c1866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:39.001563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-47c0ca02-92ef-4608-bf1d-d6162c7c1866 None None] MFA Rules not processed for user `841797717d6e4222a5d0793b3aeb3235`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:39.002750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-47c0ca02-92ef-4608-bf1d-d6162c7c1866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:39.065778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:39.098395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 673/1328] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:50:39.155861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b2bf-8a24-41b8-a580-1eb5279a2b5d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:39.155861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b2bf-8a24-41b8-a580-1eb5279a2b5d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:39.155861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b2bf-8a24-41b8-a580-1eb5279a2b5d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:39.155861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 656/1329] 10.222.0.18 () {54 vars in 964 bytes} [Tue Jun 7 15:50:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:50:39.161870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88d40-bc6d-4841-aa83-20d737dd2fb4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:39.162154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88d40-bc6d-4841-aa83-20d737dd2fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:39.162425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88d40-bc6d-4841-aa83-20d737dd2fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:39.243280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cde88d40-bc6d-4841-aa83-20d737dd2fb4 None None] MFA Rules not processed for user `dba9f1fa257a4f44ae159850f9ba349c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:39.244389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cde88d40-bc6d-4841-aa83-20d737dd2fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:39.321030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 674/1330] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Jun 7 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:39.330542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b2bf-8a24-41b8-a580-1eb5279a2b5d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:39.332056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8be4b2bf-8a24-41b8-a580-1eb5279a2b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:39.363810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:39.572458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b2bf-8a24-41b8-a580-1eb5279a2b5d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:39.646278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c476d-0cf8-427a-8be0-af17099e9e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:39.646928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c476d-0cf8-427a-8be0-af17099e9e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:39.647209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c476d-0cf8-427a-8be0-af17099e9e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:39.647483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c476d-0cf8-427a-8be0-af17099e9e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:39.649147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-697c476d-0cf8-427a-8be0-af17099e9e5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:39.729434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:39.764697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c476d-0cf8-427a-8be0-af17099e9e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:39.766267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c476d-0cf8-427a-8be0-af17099e9e5b service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:39.785250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 657/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:42.738172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2e2a-042a-457c-bc3a-79138512506d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:42.738648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2e2a-042a-457c-bc3a-79138512506d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:42.738729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2e2a-042a-457c-bc3a-79138512506d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:42.740103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 675/1332] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:42.746295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d65c7c-3da0-47f8-a736-73ccc3091b24 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:42.747054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d65c7c-3da0-47f8-a736-73ccc3091b24 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:42.747331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d65c7c-3da0-47f8-a736-73ccc3091b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:42.819105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-84d65c7c-3da0-47f8-a736-73ccc3091b24 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:42.820310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-84d65c7c-3da0-47f8-a736-73ccc3091b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:42.926587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 658/1333] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:42.962848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda2e2a-042a-457c-bc3a-79138512506d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:42.964416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bdda2e2a-042a-457c-bc3a-79138512506d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:43.038099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:43.074383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda2e2a-042a-457c-bc3a-79138512506d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:43.151523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f9023-50f2-40dd-bc29-864fb08f96e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:43.152309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f9023-50f2-40dd-bc29-864fb08f96e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:43.152709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f9023-50f2-40dd-bc29-864fb08f96e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:43.153095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f9023-50f2-40dd-bc29-864fb08f96e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:43.155222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-539f9023-50f2-40dd-bc29-864fb08f96e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:43.266588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f9023-50f2-40dd-bc29-864fb08f96e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:43.267929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f9023-50f2-40dd-bc29-864fb08f96e5 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:43.278323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:43.287387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 676/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:45.690695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bc8ad-66e9-4277-ae22-f9a501122732 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:45.691249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bc8ad-66e9-4277-ae22-f9a501122732 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:45.691249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bc8ad-66e9-4277-ae22-f9a501122732 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:45.692671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 659/1335] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:45.698013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c41d-ca8c-425f-834f-eb96838d068b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:45.698424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c41d-ca8c-425f-834f-eb96838d068b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:45.698571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c41d-ca8c-425f-834f-eb96838d068b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:45.762544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cb11c41d-ca8c-425f-834f-eb96838d068b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:45.763720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cb11c41d-ca8c-425f-834f-eb96838d068b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:45.858183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 677/1336] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:45.898165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bc8ad-66e9-4277-ae22-f9a501122732 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:45.899692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b03bc8ad-66e9-4277-ae22-f9a501122732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:45.940622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:46.007462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bc8ad-66e9-4277-ae22-f9a501122732 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:46.082006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575d2be-ac2c-472c-84c2-4ee8cc8ca3b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:46.082658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575d2be-ac2c-472c-84c2-4ee8cc8ca3b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:46.082943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575d2be-ac2c-472c-84c2-4ee8cc8ca3b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:46.083212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575d2be-ac2c-472c-84c2-4ee8cc8ca3b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:46.084873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e575d2be-ac2c-472c-84c2-4ee8cc8ca3b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:46.169177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:46.194986 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575d2be-ac2c-472c-84c2-4ee8cc8ca3b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:46.196377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575d2be-ac2c-472c-84c2-4ee8cc8ca3b1 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:46.214344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 660/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:48.936769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd92e2-7939-4251-914b-cd3ad044098e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:48.937384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd92e2-7939-4251-914b-cd3ad044098e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:48.937384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd92e2-7939-4251-914b-cd3ad044098e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:48.938720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 678/1338] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:48.944064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f4b8f-0d30-4ae1-aef2-197e02f74160 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:48.944518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f4b8f-0d30-4ae1-aef2-197e02f74160 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:48.944649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f4b8f-0d30-4ae1-aef2-197e02f74160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:49.013152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d90f4b8f-0d30-4ae1-aef2-197e02f74160 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:49.014355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d90f4b8f-0d30-4ae1-aef2-197e02f74160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:49.109471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 661/1339] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:49.145060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd92e2-7939-4251-914b-cd3ad044098e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:49.146569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b3bd92e2-7939-4251-914b-cd3ad044098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:49.223387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:49.251329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd92e2-7939-4251-914b-cd3ad044098e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:49.323027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868ccfc-12f5-4b19-88dd-52bc942980d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:49.323659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868ccfc-12f5-4b19-88dd-52bc942980d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:49.324591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868ccfc-12f5-4b19-88dd-52bc942980d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:49.324861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868ccfc-12f5-4b19-88dd-52bc942980d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:49.326468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2868ccfc-12f5-4b19-88dd-52bc942980d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:49.431992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868ccfc-12f5-4b19-88dd-52bc942980d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:49.433969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868ccfc-12f5-4b19-88dd-52bc942980d4 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:49.454119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 679/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:53.747840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7ce7-bcca-4371-9ced-cf029efbe910 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:53.748349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7ce7-bcca-4371-9ced-cf029efbe910 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:53.748349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7ce7-bcca-4371-9ced-cf029efbe910 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:53.749713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 662/1341] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:53.754996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d68d-be9c-4535-b1cf-eddba36b10cc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:53.755415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d68d-be9c-4535-b1cf-eddba36b10cc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:53.755624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d68d-be9c-4535-b1cf-eddba36b10cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:53.761790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:53.827545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7dd6d68d-be9c-4535-b1cf-eddba36b10cc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:53.828667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7dd6d68d-be9c-4535-b1cf-eddba36b10cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:53.937295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 680/1342] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:53.975542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb7ce7-bcca-4371-9ced-cf029efbe910 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:53.977938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-87eb7ce7-bcca-4371-9ced-cf029efbe910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:54.029848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:50:54.076957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb7ce7-bcca-4371-9ced-cf029efbe910 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:54.080610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c00b8-27c9-4f24-a809-13ef44e713e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:54.081199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c00b8-27c9-4f24-a809-13ef44e713e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:54.081481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c00b8-27c9-4f24-a809-13ef44e713e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:54.081755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c00b8-27c9-4f24-a809-13ef44e713e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:54.091648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c00b8-27c9-4f24-a809-13ef44e713e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:54.093106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c00b8-27c9-4f24-a809-13ef44e713e5 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:54.102986 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 663/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:50:56.996396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0862b1-f927-478a-b757-fc1a725d71cf None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:56.996907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0862b1-f927-478a-b757-fc1a725d71cf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:56.996907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0862b1-f927-478a-b757-fc1a725d71cf None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:56.998231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 681/1344] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:50:57.003490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c8ea-05e8-40e5-a7aa-610a2de04d3d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:57.003892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c8ea-05e8-40e5-a7aa-610a2de04d3d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:57.004176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c8ea-05e8-40e5-a7aa-610a2de04d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:57.032705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2952c8ea-05e8-40e5-a7aa-610a2de04d3d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:50:57.033891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2952c8ea-05e8-40e5-a7aa-610a2de04d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:50:57.111548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 664/1345] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:50:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:50:57.148739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0862b1-f927-478a-b757-fc1a725d71cf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:50:57.159793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0862b1-f927-478a-b757-fc1a725d71cf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:50:57.163395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e02708-2865-4d9a-baa8-a802573b9166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:50:57.164207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e02708-2865-4d9a-baa8-a802573b9166 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:50:57.164554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e02708-2865-4d9a-baa8-a802573b9166 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:50:57.164864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e02708-2865-4d9a-baa8-a802573b9166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:50:57.174896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e02708-2865-4d9a-baa8-a802573b9166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:50:57.176317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e02708-2865-4d9a-baa8-a802573b9166 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:50:57.186678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 682/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:50:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:01.426935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc867d-1a5a-4fc5-baa0-b79b96b2d9e9 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:01.427465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc867d-1a5a-4fc5-baa0-b79b96b2d9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:01.427465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc867d-1a5a-4fc5-baa0-b79b96b2d9e9 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:01.428948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 665/1347] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:01.434029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347007-88da-4097-a8e1-ae4b465d0256 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:01.434455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347007-88da-4097-a8e1-ae4b465d0256 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:01.434563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347007-88da-4097-a8e1-ae4b465d0256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:01.452412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-99347007-88da-4097-a8e1-ae4b465d0256 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:01.453618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-99347007-88da-4097-a8e1-ae4b465d0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:01.461502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 683/1348] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:01.500532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc867d-1a5a-4fc5-baa0-b79b96b2d9e9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:01.502211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-abcc867d-1a5a-4fc5-baa0-b79b96b2d9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:01.515082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc867d-1a5a-4fc5-baa0-b79b96b2d9e9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:01.518639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd74457-0a5d-42af-8cac-09d60f5dc19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:01.519300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd74457-0a5d-42af-8cac-09d60f5dc19e service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:01.519583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd74457-0a5d-42af-8cac-09d60f5dc19e service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:01.519861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd74457-0a5d-42af-8cac-09d60f5dc19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:01.529664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd74457-0a5d-42af-8cac-09d60f5dc19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:01.531079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd74457-0a5d-42af-8cac-09d60f5dc19e service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:01.540551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 666/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:07.015516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ba7c-4e27-41e5-8942-04ad9cab1fd4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:07.016089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ba7c-4e27-41e5-8942-04ad9cab1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:07.016310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ba7c-4e27-41e5-8942-04ad9cab1fd4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:07.018217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 684/1350] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:07.024987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f693-b4ad-4e6c-8274-2f0c6fe5d479 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:07.025799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f693-b4ad-4e6c-8274-2f0c6fe5d479 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:07.026236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f693-b4ad-4e6c-8274-2f0c6fe5d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:07.118480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f037f693-b4ad-4e6c-8274-2f0c6fe5d479 None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:07.120086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f037f693-b4ad-4e6c-8274-2f0c6fe5d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:07.149414 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:51:07.200873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 667/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:51:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:51:11.446823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=72 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:51:11.781929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad827c8-2155-4039-b83b-4e00042fa0f0 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:11.782106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad827c8-2155-4039-b83b-4e00042fa0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:11.782403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad827c8-2155-4039-b83b-4e00042fa0f0 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:11.783988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 685/1352] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:11.788936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4072-3977-4f6b-88d1-5881791645b3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:11.789444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4072-3977-4f6b-88d1-5881791645b3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:11.789762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4072-3977-4f6b-88d1-5881791645b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:11.809092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-27da4072-3977-4f6b-88d1-5881791645b3 None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:11.810428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-27da4072-3977-4f6b-88d1-5881791645b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:11.819333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 668/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:51:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:13.173793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8de4a-aaf6-4168-8225-4e99e668a84d None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:13.174288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8de4a-aaf6-4168-8225-4e99e668a84d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:13.174382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8de4a-aaf6-4168-8225-4e99e668a84d None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:13.175701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 686/1354] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:13.180678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e7d1a-71b6-42c1-b4cc-7e363e961b4a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:13.181109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e7d1a-71b6-42c1-b4cc-7e363e961b4a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:13.181249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e7d1a-71b6-42c1-b4cc-7e363e961b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:13.200477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-da3e7d1a-71b6-42c1-b4cc-7e363e961b4a None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:13.201649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-da3e7d1a-71b6-42c1-b4cc-7e363e961b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:13.209955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 669/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:17.148367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50470a9-6a05-422d-bea0-dc8082687ad5 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:17.148367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50470a9-6a05-422d-bea0-dc8082687ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:17.148367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50470a9-6a05-422d-bea0-dc8082687ad5 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:17.152132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 687/1356] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:51:17.153556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084b89f-dab8-4872-9d20-ec341f33cd0d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:17.153976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084b89f-dab8-4872-9d20-ec341f33cd0d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:17.154162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084b89f-dab8-4872-9d20-ec341f33cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:17.172060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b084b89f-dab8-4872-9d20-ec341f33cd0d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:17.173544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b084b89f-dab8-4872-9d20-ec341f33cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:17.181722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 670/1357] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:17.239363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746df82-54a9-4bca-8776-80b258fed823 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:17.239904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746df82-54a9-4bca-8776-80b258fed823 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:17.240436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746df82-54a9-4bca-8776-80b258fed823 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:17.242301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 688/1358] 10.222.0.18 () {54 vars in 961 bytes} [Tue Jun 7 15:51:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:17.252602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be2cfd-3999-4680-bd45-e3bbaf7450ea None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:17.253431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be2cfd-3999-4680-bd45-e3bbaf7450ea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:17.254100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be2cfd-3999-4680-bd45-e3bbaf7450ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:17.308181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-89be2cfd-3999-4680-bd45-e3bbaf7450ea None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:17.310559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-89be2cfd-3999-4680-bd45-e3bbaf7450ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:17.405881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 671/1359] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Jun 7 15:51:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:51:17.414467 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746df82-54a9-4bca-8776-80b258fed823 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:17.434332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746df82-54a9-4bca-8776-80b258fed823 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:17.441663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf3332-b881-42ae-9dd4-f58b3bfcb9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:17.442660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf3332-b881-42ae-9dd4-f58b3bfcb9d3 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:17.443148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf3332-b881-42ae-9dd4-f58b3bfcb9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:17.443623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf3332-b881-42ae-9dd4-f58b3bfcb9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:17.454129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf3332-b881-42ae-9dd4-f58b3bfcb9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:17.456123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf3332-b881-42ae-9dd4-f58b3bfcb9d3 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:17.469236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 689/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:19.974065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ec4a-7fa4-4359-8473-aba92f4fe231 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:19.974499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ec4a-7fa4-4359-8473-aba92f4fe231 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:19.974824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ec4a-7fa4-4359-8473-aba92f4fe231 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:19.976665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 672/1361] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:19.982204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b04e82-2e53-45f9-bc66-e4385f46e44a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:19.983065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b04e82-2e53-45f9-bc66-e4385f46e44a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:19.983409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b04e82-2e53-45f9-bc66-e4385f46e44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:20.002585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a1b04e82-2e53-45f9-bc66-e4385f46e44a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:20.003746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a1b04e82-2e53-45f9-bc66-e4385f46e44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:20.012835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 690/1362] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:20.056527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3ec4a-7fa4-4359-8473-aba92f4fe231 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:20.073803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3ec4a-7fa4-4359-8473-aba92f4fe231 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:20.080992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5f43f-2a45-4f89-9494-55ca7dfab507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:20.081711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f43f-2a45-4f89-9494-55ca7dfab507 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:20.082288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f43f-2a45-4f89-9494-55ca7dfab507 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:20.082802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f43f-2a45-4f89-9494-55ca7dfab507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:20.093243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5f43f-2a45-4f89-9494-55ca7dfab507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:20.094888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5f43f-2a45-4f89-9494-55ca7dfab507 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:20.107242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 673/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:22.761079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae13132-1a9b-420d-865e-4367b1de1afb None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:22.761595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae13132-1a9b-420d-865e-4367b1de1afb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:22.761595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae13132-1a9b-420d-865e-4367b1de1afb None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:22.762920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 691/1364] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:22.768050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb3860-4257-485b-88c7-d2e852e81c47 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:22.768456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb3860-4257-485b-88c7-d2e852e81c47 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:22.768735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb3860-4257-485b-88c7-d2e852e81c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:22.787601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9acb3860-4257-485b-88c7-d2e852e81c47 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:22.788839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9acb3860-4257-485b-88c7-d2e852e81c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:22.797255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 674/1365] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:22.843082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5672b8b-94f2-4ff8-a162-108b1f612eb8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:22.843437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5672b8b-94f2-4ff8-a162-108b1f612eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:22.843776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5672b8b-94f2-4ff8-a162-108b1f612eb8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:22.845285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 692/1366] 10.222.0.18 () {54 vars in 961 bytes} [Tue Jun 7 15:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:22.850479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7cee-d2f5-4354-ab83-6f748ff4face None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:22.850771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7cee-d2f5-4354-ab83-6f748ff4face None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:22.851048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7cee-d2f5-4354-ab83-6f748ff4face None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:22.869196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2e9b7cee-d2f5-4354-ab83-6f748ff4face None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:22.870376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2e9b7cee-d2f5-4354-ab83-6f748ff4face None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:22.879363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 675/1367] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Jun 7 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:22.887962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5672b8b-94f2-4ff8-a162-108b1f612eb8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:22.903158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5672b8b-94f2-4ff8-a162-108b1f612eb8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:22.908520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dce68b-e7b7-4788-83a7-1aa16a3a5d01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:22.909143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dce68b-e7b7-4788-83a7-1aa16a3a5d01 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:22.909421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dce68b-e7b7-4788-83a7-1aa16a3a5d01 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:22.909691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dce68b-e7b7-4788-83a7-1aa16a3a5d01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:22.918095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dce68b-e7b7-4788-83a7-1aa16a3a5d01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:22.919659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dce68b-e7b7-4788-83a7-1aa16a3a5d01 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:22.929827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 693/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:25.563551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bce3-48da-40a6-8aac-1424c00db301 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:25.564055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bce3-48da-40a6-8aac-1424c00db301 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:25.564055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bce3-48da-40a6-8aac-1424c00db301 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:25.565444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 676/1369] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:25.570849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea519a4-2e24-4557-8094-4db9b31c8642 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:25.571280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea519a4-2e24-4557-8094-4db9b31c8642 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:25.571393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd7330-581e-4348-be68-0fc8351d6750 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:25.571473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea519a4-2e24-4557-8094-4db9b31c8642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:25.571637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd7330-581e-4348-be68-0fc8351d6750 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:25.572098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd7330-581e-4348-be68-0fc8351d6750 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:25.574040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 677/1370] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:51:25.579035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1a3a4-3b89-47e2-b9c8-d7f1a30a5d5d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:25.579304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1a3a4-3b89-47e2-b9c8-d7f1a30a5d5d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:25.579739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1a3a4-3b89-47e2-b9c8-d7f1a30a5d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:25.597316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cea519a4-2e24-4557-8094-4db9b31c8642 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:25.599359 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-25f1a3a4-3b89-47e2-b9c8-d7f1a30a5d5d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:25.599498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cea519a4-2e24-4557-8094-4db9b31c8642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:25.600612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-25f1a3a4-3b89-47e2-b9c8-d7f1a30a5d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:25.610189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 678/1371] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:51:25.612153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 694/1372] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:25.666236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea519a4-2e24-4557-8094-4db9b31c8642 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:25.679832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea519a4-2e24-4557-8094-4db9b31c8642 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:25.686569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3bab0-5bd6-47ef-bb37-5e3d0f4be1ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:25.687220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bab0-5bd6-47ef-bb37-5e3d0f4be1ed service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:25.687496 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bab0-5bd6-47ef-bb37-5e3d0f4be1ed service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:25.687774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bab0-5bd6-47ef-bb37-5e3d0f4be1ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:25.696897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3bab0-5bd6-47ef-bb37-5e3d0f4be1ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:25.698906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3bab0-5bd6-47ef-bb37-5e3d0f4be1ed service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:25.709471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 695/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:25.936222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f262-aa62-4ff0-9454-48feb7b925dd None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:25.936745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f262-aa62-4ff0-9454-48feb7b925dd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:25.937050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f262-aa62-4ff0-9454-48feb7b925dd None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:25.940343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 679/1374] 10.222.0.18 () {54 vars in 963 bytes} [Tue Jun 7 15:51:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:25.945063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbdcaf-198d-41e1-a273-ae3457cdecd4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:25.945423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbdcaf-198d-41e1-a273-ae3457cdecd4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:25.945646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbdcaf-198d-41e1-a273-ae3457cdecd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:26.006458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-fbcbdcaf-198d-41e1-a273-ae3457cdecd4 None None] MFA Rules not processed for user `a31ce1720fa84c43aec2ea016371f475`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:26.007661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fbcbdcaf-198d-41e1-a273-ae3457cdecd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:26.071489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 696/1375] 10.222.0.18 () {58 vars in 1060 bytes} [Tue Jun 7 15:51:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:26.079059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f262-aa62-4ff0-9454-48feb7b925dd None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:26.090321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f262-aa62-4ff0-9454-48feb7b925dd None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:26.093858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c0bb9-ee43-4965-a899-a26e48d64517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:26.094498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c0bb9-ee43-4965-a899-a26e48d64517 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:26.094782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c0bb9-ee43-4965-a899-a26e48d64517 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:26.095059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c0bb9-ee43-4965-a899-a26e48d64517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:26.105543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c0bb9-ee43-4965-a899-a26e48d64517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:26.106975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c0bb9-ee43-4965-a899-a26e48d64517 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:26.116941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 680/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:28.116975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41167e-7339-4bb2-9204-1b5b2d2a279b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:28.117522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41167e-7339-4bb2-9204-1b5b2d2a279b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:28.117522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41167e-7339-4bb2-9204-1b5b2d2a279b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:28.118917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 697/1377] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:28.124506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf38ef-af2a-4fe5-aba7-9ab25e813d1e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:28.124948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf38ef-af2a-4fe5-aba7-9ab25e813d1e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:28.125064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf38ef-af2a-4fe5-aba7-9ab25e813d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:28.132057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:51:28.144730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-91bf38ef-af2a-4fe5-aba7-9ab25e813d1e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:28.145936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-91bf38ef-af2a-4fe5-aba7-9ab25e813d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:28.157682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 681/1378] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:28.202309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41167e-7339-4bb2-9204-1b5b2d2a279b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:28.219150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41167e-7339-4bb2-9204-1b5b2d2a279b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:28.225088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbf82c-6c4a-470f-a376-a661ab9253bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:28.225735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf82c-6c4a-470f-a376-a661ab9253bf service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:28.226017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf82c-6c4a-470f-a376-a661ab9253bf service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:28.226290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf82c-6c4a-470f-a376-a661ab9253bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:28.235493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbf82c-6c4a-470f-a376-a661ab9253bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:28.237121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbf82c-6c4a-470f-a376-a661ab9253bf service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:28.248181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 698/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:30.574477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885eea9-835b-4ab1-a778-42674beed9b0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:30.574993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885eea9-835b-4ab1-a778-42674beed9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:30.575197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885eea9-835b-4ab1-a778-42674beed9b0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:30.577251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 682/1380] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:30.581980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f1180-0539-4a3e-afee-13b311fe8708 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:30.582394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f1180-0539-4a3e-afee-13b311fe8708 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:30.582535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f1180-0539-4a3e-afee-13b311fe8708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:30.601697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a32f1180-0539-4a3e-afee-13b311fe8708 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:30.602853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a32f1180-0539-4a3e-afee-13b311fe8708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:30.611923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 699/1381] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:30.655506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885eea9-835b-4ab1-a778-42674beed9b0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:30.671230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885eea9-835b-4ab1-a778-42674beed9b0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:30.676624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb3e90-030e-4602-a37a-63db4853d32c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:30.677260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb3e90-030e-4602-a37a-63db4853d32c service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:30.677542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb3e90-030e-4602-a37a-63db4853d32c service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:30.677821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb3e90-030e-4602-a37a-63db4853d32c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:30.686626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb3e90-030e-4602-a37a-63db4853d32c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:30.688185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb3e90-030e-4602-a37a-63db4853d32c service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:30.698667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 683/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:33.046458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8afae-20b2-41f5-b27a-7045f8c59bd0 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:33.047034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8afae-20b2-41f5-b27a-7045f8c59bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:33.047034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8afae-20b2-41f5-b27a-7045f8c59bd0 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:33.051951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 700/1383] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:33.053623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e931051-2a36-4530-ab56-6c25255e3ef8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:33.054020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e931051-2a36-4530-ab56-6c25255e3ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:33.054172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e931051-2a36-4530-ab56-6c25255e3ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:33.074187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8e931051-2a36-4530-ab56-6c25255e3ef8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:33.075366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e931051-2a36-4530-ab56-6c25255e3ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:33.083610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 684/1384] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:51:33.126795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8afae-20b2-41f5-b27a-7045f8c59bd0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:33.147421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8afae-20b2-41f5-b27a-7045f8c59bd0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:33.154254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d76ccb-049f-4e28-815f-a002a90355c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:33.155268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76ccb-049f-4e28-815f-a002a90355c9 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:33.155765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76ccb-049f-4e28-815f-a002a90355c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:33.156256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76ccb-049f-4e28-815f-a002a90355c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:33.165854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d76ccb-049f-4e28-815f-a002a90355c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:33.168402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d76ccb-049f-4e28-815f-a002a90355c9 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:33.187106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 701/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:35.544943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337f2a-6cab-41d7-887e-e25b626daa95 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:35.545658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337f2a-6cab-41d7-887e-e25b626daa95 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:35.545658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337f2a-6cab-41d7-887e-e25b626daa95 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:35.546826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 685/1386] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:35.552875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1e5e2-8f77-40c4-b29f-bf31bcdbb881 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:35.553356 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1e5e2-8f77-40c4-b29f-bf31bcdbb881 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:35.553623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1e5e2-8f77-40c4-b29f-bf31bcdbb881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:35.578105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-16d1e5e2-8f77-40c4-b29f-bf31bcdbb881 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:35.579994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-16d1e5e2-8f77-40c4-b29f-bf31bcdbb881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:35.592035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 702/1387] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:35.636545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337f2a-6cab-41d7-887e-e25b626daa95 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:35.651868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337f2a-6cab-41d7-887e-e25b626daa95 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:35.657416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b625c8-d7d9-421a-9be9-a8b673591f0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:35.658051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b625c8-d7d9-421a-9be9-a8b673591f0f service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:35.658339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b625c8-d7d9-421a-9be9-a8b673591f0f service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:35.658612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b625c8-d7d9-421a-9be9-a8b673591f0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:35.666999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b625c8-d7d9-421a-9be9-a8b673591f0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:35.668602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b625c8-d7d9-421a-9be9-a8b673591f0f service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:35.678183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 686/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:37.965112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63174f-7af6-4cb7-9215-fbb8ff20a903 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:37.965675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63174f-7af6-4cb7-9215-fbb8ff20a903 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:37.965825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63174f-7af6-4cb7-9215-fbb8ff20a903 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:37.967727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 703/1389] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:37.972898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812b759-344a-481c-89b8-d75cfba5cd5d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:37.973292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812b759-344a-481c-89b8-d75cfba5cd5d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:37.973452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812b759-344a-481c-89b8-d75cfba5cd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:37.996373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d812b759-344a-481c-89b8-d75cfba5cd5d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:37.997878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d812b759-344a-481c-89b8-d75cfba5cd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:38.007871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 687/1390] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:38.053250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63174f-7af6-4cb7-9215-fbb8ff20a903 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:38.072015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63174f-7af6-4cb7-9215-fbb8ff20a903 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:38.078690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dbd94-48be-4690-a49a-3210a50c96a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:38.079800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dbd94-48be-4690-a49a-3210a50c96a5 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:38.080151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dbd94-48be-4690-a49a-3210a50c96a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:38.080527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dbd94-48be-4690-a49a-3210a50c96a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:38.091616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dbd94-48be-4690-a49a-3210a50c96a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:38.093661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dbd94-48be-4690-a49a-3210a50c96a5 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:38.106949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 704/1391] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:40.686168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf86ab-af0d-4b91-892f-2a199d250aea None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:40.686791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf86ab-af0d-4b91-892f-2a199d250aea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:40.686791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf86ab-af0d-4b91-892f-2a199d250aea None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:40.688086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 688/1392] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:40.693254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86af2-8d59-431e-8fb8-475db057fe7e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:40.693692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86af2-8d59-431e-8fb8-475db057fe7e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:40.693802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86af2-8d59-431e-8fb8-475db057fe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:40.711132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-14e86af2-8d59-431e-8fb8-475db057fe7e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:40.712333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-14e86af2-8d59-431e-8fb8-475db057fe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:40.720179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 705/1393] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:40.764685 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf86ab-af0d-4b91-892f-2a199d250aea None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:40.779324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf86ab-af0d-4b91-892f-2a199d250aea None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:40.784720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212f5d9-0906-4a3d-9ea2-efa0dd18932f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:40.785354 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212f5d9-0906-4a3d-9ea2-efa0dd18932f service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:40.785636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212f5d9-0906-4a3d-9ea2-efa0dd18932f service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:40.786047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212f5d9-0906-4a3d-9ea2-efa0dd18932f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:40.794756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212f5d9-0906-4a3d-9ea2-efa0dd18932f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:40.796343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212f5d9-0906-4a3d-9ea2-efa0dd18932f service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:40.805596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 689/1394] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:42.105929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d28cb-1075-4595-a7d3-93df7ff73865 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:42.106494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d28cb-1075-4595-a7d3-93df7ff73865 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:42.106494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d28cb-1075-4595-a7d3-93df7ff73865 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:42.107868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 706/1395] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:42.113357 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb86343-9b2d-4d2d-8d6e-49a262954c8d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:42.113841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb86343-9b2d-4d2d-8d6e-49a262954c8d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:42.113978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb86343-9b2d-4d2d-8d6e-49a262954c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:42.132801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0bb86343-9b2d-4d2d-8d6e-49a262954c8d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:42.133980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0bb86343-9b2d-4d2d-8d6e-49a262954c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:42.142386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 690/1396] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:42.199324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16811ffd-1ee0-4957-a3e9-8a7ac25a9444 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:42.199703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16811ffd-1ee0-4957-a3e9-8a7ac25a9444 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:42.200051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16811ffd-1ee0-4957-a3e9-8a7ac25a9444 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:42.201545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 707/1397] 10.222.0.18 () {54 vars in 962 bytes} [Tue Jun 7 15:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:42.206723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e94c-7129-4dd2-9454-0aa4974c43fd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:42.207044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e94c-7129-4dd2-9454-0aa4974c43fd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:42.207324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e94c-7129-4dd2-9454-0aa4974c43fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:42.255682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-40d6e94c-7129-4dd2-9454-0aa4974c43fd None None] MFA Rules not processed for user `ac0206e09e4f426d9f4b54543d89c039`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:42.256844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-40d6e94c-7129-4dd2-9454-0aa4974c43fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:42.316569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 691/1398] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Jun 7 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:42.324646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16811ffd-1ee0-4957-a3e9-8a7ac25a9444 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:42.330156 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:51:42.336501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16811ffd-1ee0-4957-a3e9-8a7ac25a9444 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:42.340127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec514d7-0135-4d3c-bfa8-71544e8d1527 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0206e09e4f426d9f4b54543d89c039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:42.340798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec514d7-0135-4d3c-bfa8-71544e8d1527 service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:42.341080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec514d7-0135-4d3c-bfa8-71544e8d1527 service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:42.341356 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec514d7-0135-4d3c-bfa8-71544e8d1527 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:42.350400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec514d7-0135-4d3c-bfa8-71544e8d1527 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:42.352064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec514d7-0135-4d3c-bfa8-71544e8d1527 service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:42.362214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 708/1399] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:42.901502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494bc8b-353c-450b-b8ac-91c5e7c677f5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:42.901740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494bc8b-353c-450b-b8ac-91c5e7c677f5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:42.902016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494bc8b-353c-450b-b8ac-91c5e7c677f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:42.950143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d494bc8b-353c-450b-b8ac-91c5e7c677f5 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:42.951326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d494bc8b-353c-450b-b8ac-91c5e7c677f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:42.996033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:51:43.017728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 692/1400] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:43.059389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bfd7e-792c-4f85-a5ba-a55f5c4e00b6 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:43.059762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bfd7e-792c-4f85-a5ba-a55f5c4e00b6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:43.060102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bfd7e-792c-4f85-a5ba-a55f5c4e00b6 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:43.061603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 709/1401] 10.222.0.18 () {54 vars in 942 bytes} [Tue Jun 7 15:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:43.067862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e262820-1fc4-4bb2-b4ed-8463a4e4f6bd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:43.068149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e262820-1fc4-4bb2-b4ed-8463a4e4f6bd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:43.068457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e262820-1fc4-4bb2-b4ed-8463a4e4f6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:43.122517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5e262820-1fc4-4bb2-b4ed-8463a4e4f6bd None None] MFA Rules not processed for user `31137a8197ab4072b47646b893a9733a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:43.123696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5e262820-1fc4-4bb2-b4ed-8463a4e4f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:43.137224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdd0db-0a8d-4d32-b26c-9e89dc10fcca None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:43.137744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdd0db-0a8d-4d32-b26c-9e89dc10fcca None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:43.137744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdd0db-0a8d-4d32-b26c-9e89dc10fcca None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:43.139186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 710/1402] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:43.144056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800a00f-e65d-4a9f-a609-41475db29383 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:43.144340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800a00f-e65d-4a9f-a609-41475db29383 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:43.144849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800a00f-e65d-4a9f-a609-41475db29383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:43.162617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7800a00f-e65d-4a9f-a609-41475db29383 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:43.163843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7800a00f-e65d-4a9f-a609-41475db29383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:43.176810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 711/1403] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:51:43.209579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 693/1404] 10.222.0.18 () {58 vars in 1039 bytes} [Tue Jun 7 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:43.220974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800a00f-e65d-4a9f-a609-41475db29383 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:43.222328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e262820-1fc4-4bb2-b4ed-8463a4e4f6bd None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:43.234392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800a00f-e65d-4a9f-a609-41475db29383 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:43.235652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e262820-1fc4-4bb2-b4ed-8463a4e4f6bd None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:43.239422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43636968-1a0b-4af3-b6bc-f17f49f60c55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:43.240068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43636968-1a0b-4af3-b6bc-f17f49f60c55 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:43.240348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43636968-1a0b-4af3-b6bc-f17f49f60c55 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:43.240659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43636968-1a0b-4af3-b6bc-f17f49f60c55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:43.240971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c59b6c-08c0-4c58-a226-ab267a720f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:43.241603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c59b6c-08c0-4c58-a226-ab267a720f69 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:43.241879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c59b6c-08c0-4c58-a226-ab267a720f69 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:43.242151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c59b6c-08c0-4c58-a226-ab267a720f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:43.249353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43636968-1a0b-4af3-b6bc-f17f49f60c55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:43.250916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43636968-1a0b-4af3-b6bc-f17f49f60c55 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:43.251025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c59b6c-08c0-4c58-a226-ab267a720f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:43.252569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c59b6c-08c0-4c58-a226-ab267a720f69 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:43.259005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 712/1405] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 15:51:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 15:51:43.262970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 694/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:51:45.604247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9dbb-801f-4b2b-a873-315f58d15f99 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:45.604772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9dbb-801f-4b2b-a873-315f58d15f99 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:45.605215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c9dbb-801f-4b2b-a873-315f58d15f99 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:45.607355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 713/1407] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:51:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:51:45.613168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8bc55-9cf1-4f4d-99f2-c671457b821a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:45.614371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8bc55-9cf1-4f4d-99f2-c671457b821a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:45.614923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8bc55-9cf1-4f4d-99f2-c671457b821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:45.634468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f5c8bc55-9cf1-4f4d-99f2-c671457b821a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:51:45.636525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f5c8bc55-9cf1-4f4d-99f2-c671457b821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:51:45.644953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 695/1408] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:51:45.690187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9dbb-801f-4b2b-a873-315f58d15f99 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:51:45.705826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c9dbb-801f-4b2b-a873-315f58d15f99 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:51:45.717772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c8570-c756-4133-9a00-e88bde54b7a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:51:45.718430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c8570-c756-4133-9a00-e88bde54b7a0 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:51:45.718711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c8570-c756-4133-9a00-e88bde54b7a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:51:45.718987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c8570-c756-4133-9a00-e88bde54b7a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:51:45.729198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c8570-c756-4133-9a00-e88bde54b7a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:51:45.730818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c8570-c756-4133-9a00-e88bde54b7a0 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:51:45.742157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 714/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:51:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:51:55.637131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=59 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:51:55.736747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=20 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:52:07.290599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaec8d-61af-420a-a487-ceebb1406a70 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:07.291188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaec8d-61af-420a-a487-ceebb1406a70 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:07.291188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaec8d-61af-420a-a487-ceebb1406a70 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:07.292527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 696/1410] 10.222.0.18 () {54 vars in 918 bytes} [Tue Jun 7 15:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:07.297458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab126c-dc3b-45d5-a3ab-897fdeb66c3f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:07.297891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab126c-dc3b-45d5-a3ab-897fdeb66c3f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:07.298006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab126c-dc3b-45d5-a3ab-897fdeb66c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:07.316014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7fab126c-dc3b-45d5-a3ab-897fdeb66c3f None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:07.317218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7fab126c-dc3b-45d5-a3ab-897fdeb66c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:07.326352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 715/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 7 15:52:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:07.340173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd298a7-806a-400c-993b-5c888e052716 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:07.340492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd298a7-806a-400c-993b-5c888e052716 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:07.340771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd298a7-806a-400c-993b-5c888e052716 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:07.342102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 697/1412] 10.222.0.18 () {54 vars in 918 bytes} [Tue Jun 7 15:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:52:07.346636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aae226-2cec-45a8-bd5b-288781ce2bec None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:07.346919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aae226-2cec-45a8-bd5b-288781ce2bec None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:07.347193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aae226-2cec-45a8-bd5b-288781ce2bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:07.365800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-53aae226-2cec-45a8-bd5b-288781ce2bec None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:07.366994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-53aae226-2cec-45a8-bd5b-288781ce2bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:07.377166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 716/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 7 15:52:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:10.104749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a669e2-1b05-44c9-ad2e-5e8d922b8397 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:10.105295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a669e2-1b05-44c9-ad2e-5e8d922b8397 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:10.105295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a669e2-1b05-44c9-ad2e-5e8d922b8397 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:10.106626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:10.111778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379def1a-9164-4b70-84a1-6a5f5727562e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:10.112201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379def1a-9164-4b70-84a1-6a5f5727562e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:10.112348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379def1a-9164-4b70-84a1-6a5f5727562e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:10.129792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-379def1a-9164-4b70-84a1-6a5f5727562e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:10.130991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-379def1a-9164-4b70-84a1-6a5f5727562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:10.139482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:10.182999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a669e2-1b05-44c9-ad2e-5e8d922b8397 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:10.198838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a669e2-1b05-44c9-ad2e-5e8d922b8397 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:10.204099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d23f88-c5f3-4fba-942f-df8f18477cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:10.204750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d23f88-c5f3-4fba-942f-df8f18477cfb service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:10.205031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d23f88-c5f3-4fba-942f-df8f18477cfb service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:10.205304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d23f88-c5f3-4fba-942f-df8f18477cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:10.214167 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d23f88-c5f3-4fba-942f-df8f18477cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:10.215727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d23f88-c5f3-4fba-942f-df8f18477cfb service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:10.225045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 699/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:11.203489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e577-3878-4020-ad5c-c8c67e768613 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:11.203936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e577-3878-4020-ad5c-c8c67e768613 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:11.204062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e577-3878-4020-ad5c-c8c67e768613 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:11.205468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:11.210253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015baa13-83ec-46bf-9072-b6c9c18bd1eb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:11.210676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015baa13-83ec-46bf-9072-b6c9c18bd1eb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:11.210838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015baa13-83ec-46bf-9072-b6c9c18bd1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:11.230029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-015baa13-83ec-46bf-9072-b6c9c18bd1eb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:11.231219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-015baa13-83ec-46bf-9072-b6c9c18bd1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:11.238972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:11.410176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021e577-3878-4020-ad5c-c8c67e768613 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:11.425102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021e577-3878-4020-ad5c-c8c67e768613 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:11.430328 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec0fbe-243f-4ad4-bcca-5d6f21ca2798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:11.431793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec0fbe-243f-4ad4-bcca-5d6f21ca2798 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:11.432087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec0fbe-243f-4ad4-bcca-5d6f21ca2798 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:11.432378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec0fbe-243f-4ad4-bcca-5d6f21ca2798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:11.440338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec0fbe-243f-4ad4-bcca-5d6f21ca2798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:11.441903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec0fbe-243f-4ad4-bcca-5d6f21ca2798 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:11.451443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 719/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:12.425743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b893-eef7-4648-b7b9-3c7e3debab29 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:12.426305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b893-eef7-4648-b7b9-3c7e3debab29 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:12.426305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b893-eef7-4648-b7b9-3c7e3debab29 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:12.427682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:12.433369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1ef86-ad0e-4db4-a3c2-4f8ebc5d7f65 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:12.433763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1ef86-ad0e-4db4-a3c2-4f8ebc5d7f65 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:12.433929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1ef86-ad0e-4db4-a3c2-4f8ebc5d7f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:12.451630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-75d1ef86-ad0e-4db4-a3c2-4f8ebc5d7f65 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:12.452853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-75d1ef86-ad0e-4db4-a3c2-4f8ebc5d7f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:12.460615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:12.611501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859b893-eef7-4648-b7b9-3c7e3debab29 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:12.625687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859b893-eef7-4648-b7b9-3c7e3debab29 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:12.630879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b82f3-78c7-4086-9c8b-a6715d154028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:12.631499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b82f3-78c7-4086-9c8b-a6715d154028 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:12.631776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b82f3-78c7-4086-9c8b-a6715d154028 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:12.632047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b82f3-78c7-4086-9c8b-a6715d154028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:12.640533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b82f3-78c7-4086-9c8b-a6715d154028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:12.642085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b82f3-78c7-4086-9c8b-a6715d154028 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:12.652156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 702/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:13.551386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644355-60e8-429f-9ae3-42dc5806963e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:13.551929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644355-60e8-429f-9ae3-42dc5806963e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:13.551929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644355-60e8-429f-9ae3-42dc5806963e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:13.553408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:13.560467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854624c-af81-4699-aa56-45ebe6fc1972 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:13.560915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854624c-af81-4699-aa56-45ebe6fc1972 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:13.561028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854624c-af81-4699-aa56-45ebe6fc1972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:13.579804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c854624c-af81-4699-aa56-45ebe6fc1972 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:13.581028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c854624c-af81-4699-aa56-45ebe6fc1972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:13.589677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:13.738997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f644355-60e8-429f-9ae3-42dc5806963e None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:13.753732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f644355-60e8-429f-9ae3-42dc5806963e None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:13.759043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e3c2c-ccf1-401c-9c20-b671f6be949a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:13.759661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3c2c-ccf1-401c-9c20-b671f6be949a service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:13.759938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3c2c-ccf1-401c-9c20-b671f6be949a service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:13.760210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3c2c-ccf1-401c-9c20-b671f6be949a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:13.768132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e3c2c-ccf1-401c-9c20-b671f6be949a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:13.769715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e3c2c-ccf1-401c-9c20-b671f6be949a service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:13.779046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 722/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:14.692987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f956-45e0-4692-87ad-8ff0f34732ac None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:14.693546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f956-45e0-4692-87ad-8ff0f34732ac None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:14.693546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f956-45e0-4692-87ad-8ff0f34732ac None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:14.694915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:14.700320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0633a17-d647-4bae-bba3-b5fe28c07921 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:14.700787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0633a17-d647-4bae-bba3-b5fe28c07921 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:14.700885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0633a17-d647-4bae-bba3-b5fe28c07921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:14.718980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e0633a17-d647-4bae-bba3-b5fe28c07921 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:14.720163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e0633a17-d647-4bae-bba3-b5fe28c07921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:14.728161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:14.878073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094f956-45e0-4692-87ad-8ff0f34732ac None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:14.893857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094f956-45e0-4692-87ad-8ff0f34732ac None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:14.899192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be6deb-344c-4212-a5ac-5c1a14becb89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:14.899821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be6deb-344c-4212-a5ac-5c1a14becb89 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:14.900101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be6deb-344c-4212-a5ac-5c1a14becb89 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:14.900407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be6deb-344c-4212-a5ac-5c1a14becb89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:14.908333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be6deb-344c-4212-a5ac-5c1a14becb89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:14.909928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be6deb-344c-4212-a5ac-5c1a14becb89 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:14.919289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 705/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:15.764996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ca98-d0f5-4bc3-8e1e-870ce2dcc01c None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:15.765536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ca98-d0f5-4bc3-8e1e-870ce2dcc01c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:15.765536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ca98-d0f5-4bc3-8e1e-870ce2dcc01c None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:15.766926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:15.771735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f72596-8112-451e-9ca9-87702ea4aa0a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:15.772168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f72596-8112-451e-9ca9-87702ea4aa0a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:15.772322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f72596-8112-451e-9ca9-87702ea4aa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:15.791270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-74f72596-8112-451e-9ca9-87702ea4aa0a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:15.792488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-74f72596-8112-451e-9ca9-87702ea4aa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:15.800213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:15.950571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ca98-d0f5-4bc3-8e1e-870ce2dcc01c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:15.965521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ca98-d0f5-4bc3-8e1e-870ce2dcc01c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:15.970951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b9d8b-1e4b-4998-b12a-7513183dca38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:15.971577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b9d8b-1e4b-4998-b12a-7513183dca38 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:15.971856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b9d8b-1e4b-4998-b12a-7513183dca38 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:15.972131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b9d8b-1e4b-4998-b12a-7513183dca38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:15.980378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b9d8b-1e4b-4998-b12a-7513183dca38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:15.981999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b9d8b-1e4b-4998-b12a-7513183dca38 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:15.991270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 725/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:16.839647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8077595-fbb7-4595-bcb2-316fc74b8b47 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:16.840160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8077595-fbb7-4595-bcb2-316fc74b8b47 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:16.840257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8077595-fbb7-4595-bcb2-316fc74b8b47 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:16.841610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:16.847977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9162cf-075f-40a8-925c-c51c165bfe02 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:16.848453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9162cf-075f-40a8-925c-c51c165bfe02 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:16.848557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9162cf-075f-40a8-925c-c51c165bfe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:16.867474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4d9162cf-075f-40a8-925c-c51c165bfe02 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:16.869241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d9162cf-075f-40a8-925c-c51c165bfe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:16.877306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:17.040231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8077595-fbb7-4595-bcb2-316fc74b8b47 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:17.054233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8077595-fbb7-4595-bcb2-316fc74b8b47 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:17.060319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41b7a7-b458-4b90-9286-a1ae0480fead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:17.060974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41b7a7-b458-4b90-9286-a1ae0480fead service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:17.061257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41b7a7-b458-4b90-9286-a1ae0480fead service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:17.061532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41b7a7-b458-4b90-9286-a1ae0480fead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:17.069599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41b7a7-b458-4b90-9286-a1ae0480fead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:17.072016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41b7a7-b458-4b90-9286-a1ae0480fead service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:17.081377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 708/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:17.932144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a265-07fc-4ee0-98a9-8f0ab0f382cb None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:17.932744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a265-07fc-4ee0-98a9-8f0ab0f382cb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:17.932744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a265-07fc-4ee0-98a9-8f0ab0f382cb None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:17.934140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:17.939026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6f14c-aba0-43f3-845d-9d305f04aeec None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:17.939478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6f14c-aba0-43f3-845d-9d305f04aeec None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:17.939587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6f14c-aba0-43f3-845d-9d305f04aeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:17.957253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-59b6f14c-aba0-43f3-845d-9d305f04aeec None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:17.958429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-59b6f14c-aba0-43f3-845d-9d305f04aeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:17.966529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:18.115550 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6a265-07fc-4ee0-98a9-8f0ab0f382cb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:18.130791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6a265-07fc-4ee0-98a9-8f0ab0f382cb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:18.136936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a025ae-a141-40cd-adac-d42cef3eb21e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:18.137568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a025ae-a141-40cd-adac-d42cef3eb21e service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:18.137847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a025ae-a141-40cd-adac-d42cef3eb21e service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:18.138120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a025ae-a141-40cd-adac-d42cef3eb21e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:18.146668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a025ae-a141-40cd-adac-d42cef3eb21e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:18.148235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a025ae-a141-40cd-adac-d42cef3eb21e service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:18.157886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 728/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:19.004759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf8046-d470-4982-a7cd-85cb03f0a53d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:19.005202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf8046-d470-4982-a7cd-85cb03f0a53d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:19.005319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf8046-d470-4982-a7cd-85cb03f0a53d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:19.006668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:19.012035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e65e-0c12-4298-9973-a8e9bd2538e2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:19.012450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e65e-0c12-4298-9973-a8e9bd2538e2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:19.012608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e65e-0c12-4298-9973-a8e9bd2538e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:19.030803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6d08e65e-0c12-4298-9973-a8e9bd2538e2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:19.031975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6d08e65e-0c12-4298-9973-a8e9bd2538e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:19.039555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:19.188539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf8046-d470-4982-a7cd-85cb03f0a53d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:19.202873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf8046-d470-4982-a7cd-85cb03f0a53d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:19.208125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e88a7-ca20-4223-9557-bf599735efa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:19.208783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e88a7-ca20-4223-9557-bf599735efa3 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:19.209068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e88a7-ca20-4223-9557-bf599735efa3 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:19.209342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e88a7-ca20-4223-9557-bf599735efa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:19.217900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e88a7-ca20-4223-9557-bf599735efa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:19.219474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e88a7-ca20-4223-9557-bf599735efa3 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:19.229659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 711/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:21.353467 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69c8ce-1a42-45b9-882d-db67d00926f8 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:21.354040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69c8ce-1a42-45b9-882d-db67d00926f8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:21.354040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69c8ce-1a42-45b9-882d-db67d00926f8 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:21.355493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:21.361056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609741a9-a387-43ae-994a-328638da6602 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:21.361487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609741a9-a387-43ae-994a-328638da6602 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:21.361611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609741a9-a387-43ae-994a-328638da6602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:21.379108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-609741a9-a387-43ae-994a-328638da6602 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:21.380306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-609741a9-a387-43ae-994a-328638da6602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:21.388957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:21.427623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69c8ce-1a42-45b9-882d-db67d00926f8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:21.438486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69c8ce-1a42-45b9-882d-db67d00926f8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:21.442005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa78c3-e0a7-4515-8cec-6529d935c195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:21.442638 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa78c3-e0a7-4515-8cec-6529d935c195 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:21.442922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa78c3-e0a7-4515-8cec-6529d935c195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:21.443199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa78c3-e0a7-4515-8cec-6529d935c195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:21.453317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa78c3-e0a7-4515-8cec-6529d935c195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:21.454747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa78c3-e0a7-4515-8cec-6529d935c195 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:21.464848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 731/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:23.950738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ff853-4bdd-4992-aa9c-03e2c66b6c46 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:23.951279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ff853-4bdd-4992-aa9c-03e2c66b6c46 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:23.951279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ff853-4bdd-4992-aa9c-03e2c66b6c46 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:23.952690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:23.957824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4a35-4bcb-42a6-b640-226a4d9736d2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:23.958228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4a35-4bcb-42a6-b640-226a4d9736d2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:23.958377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4a35-4bcb-42a6-b640-226a4d9736d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:23.977991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-af5b4a35-4bcb-42a6-b640-226a4d9736d2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:23.979177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-af5b4a35-4bcb-42a6-b640-226a4d9736d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:23.986881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:24.025755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ff853-4bdd-4992-aa9c-03e2c66b6c46 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:24.036406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ff853-4bdd-4992-aa9c-03e2c66b6c46 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:24.040671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c796a9-a009-4ef8-9eac-3f49b832d53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:24.041300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c796a9-a009-4ef8-9eac-3f49b832d53a service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:24.041583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c796a9-a009-4ef8-9eac-3f49b832d53a service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:24.041855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c796a9-a009-4ef8-9eac-3f49b832d53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:24.050619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c796a9-a009-4ef8-9eac-3f49b832d53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:24.052039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c796a9-a009-4ef8-9eac-3f49b832d53a service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:24.061421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 714/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:26.647607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855eb66-ab24-4d55-bf77-6e2637774412 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:26.648129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855eb66-ab24-4d55-bf77-6e2637774412 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:26.648243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855eb66-ab24-4d55-bf77-6e2637774412 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:26.650299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:26.655491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5705a2-704e-494d-8ad5-eb3a1cbe3e80 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:26.655897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5705a2-704e-494d-8ad5-eb3a1cbe3e80 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:26.656080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5705a2-704e-494d-8ad5-eb3a1cbe3e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:26.673453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8e5705a2-704e-494d-8ad5-eb3a1cbe3e80 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:26.674634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e5705a2-704e-494d-8ad5-eb3a1cbe3e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:26.682322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:26.720864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855eb66-ab24-4d55-bf77-6e2637774412 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:26.732109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855eb66-ab24-4d55-bf77-6e2637774412 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:26.735741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32312838-f53b-4bb5-ae21-1cc020d719ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:26.736399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32312838-f53b-4bb5-ae21-1cc020d719ef service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:26.736680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32312838-f53b-4bb5-ae21-1cc020d719ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:26.736949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32312838-f53b-4bb5-ae21-1cc020d719ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:26.746541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32312838-f53b-4bb5-ae21-1cc020d719ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:26.747946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32312838-f53b-4bb5-ae21-1cc020d719ef service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:26.757281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 734/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:29.142983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f2aec-d241-4a47-9c56-fb681a94b5ab None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:29.143576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f2aec-d241-4a47-9c56-fb681a94b5ab None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:29.143576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f2aec-d241-4a47-9c56-fb681a94b5ab None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:29.144944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:29.150765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37785e0-a7de-47ea-a5a1-7e7eefae7e23 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:29.151273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37785e0-a7de-47ea-a5a1-7e7eefae7e23 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:29.151846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37785e0-a7de-47ea-a5a1-7e7eefae7e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:29.170835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f37785e0-a7de-47ea-a5a1-7e7eefae7e23 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:29.172024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f37785e0-a7de-47ea-a5a1-7e7eefae7e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:29.179868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:29.219543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f2aec-d241-4a47-9c56-fb681a94b5ab None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:29.230911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f2aec-d241-4a47-9c56-fb681a94b5ab None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:29.234343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0aae6-69d7-4f14-94ad-751e9c6f6a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:29.234972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0aae6-69d7-4f14-94ad-751e9c6f6a0f service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:29.235250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0aae6-69d7-4f14-94ad-751e9c6f6a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:29.235520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0aae6-69d7-4f14-94ad-751e9c6f6a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:29.244249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0aae6-69d7-4f14-94ad-751e9c6f6a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:29.245683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0aae6-69d7-4f14-94ad-751e9c6f6a0f service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:29.254853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 717/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:31.596844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491d2f-08da-4ab2-ab1e-a3639fe1ea07 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:31.597387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491d2f-08da-4ab2-ab1e-a3639fe1ea07 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:31.597387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491d2f-08da-4ab2-ab1e-a3639fe1ea07 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:31.598684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:31.605035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d16edc-17a1-4fa0-8e98-9711e156ed27 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:31.605493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d16edc-17a1-4fa0-8e98-9711e156ed27 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:31.605598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d16edc-17a1-4fa0-8e98-9711e156ed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:31.624981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f7d16edc-17a1-4fa0-8e98-9711e156ed27 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:31.626172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f7d16edc-17a1-4fa0-8e98-9711e156ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:31.633879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:31.672179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491d2f-08da-4ab2-ab1e-a3639fe1ea07 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:31.683430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491d2f-08da-4ab2-ab1e-a3639fe1ea07 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:31.686940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9a670-3db9-4dc2-bc2a-273f7f24c635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:31.687569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9a670-3db9-4dc2-bc2a-273f7f24c635 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:31.687851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9a670-3db9-4dc2-bc2a-273f7f24c635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:31.688127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9a670-3db9-4dc2-bc2a-273f7f24c635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:31.697215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9a670-3db9-4dc2-bc2a-273f7f24c635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:31.698687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9a670-3db9-4dc2-bc2a-273f7f24c635 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:31.708080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 737/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:34.013677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84915cd-9545-48a7-bb4b-c9733cc6bcac None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:34.014091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84915cd-9545-48a7-bb4b-c9733cc6bcac None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:34.014232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84915cd-9545-48a7-bb4b-c9733cc6bcac None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:34.015603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:34.020843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af041c46-c9b6-4bee-a224-ba38f06d2a8a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:34.021286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af041c46-c9b6-4bee-a224-ba38f06d2a8a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:34.021397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af041c46-c9b6-4bee-a224-ba38f06d2a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:34.038635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-af041c46-c9b6-4bee-a224-ba38f06d2a8a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:34.040331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-af041c46-c9b6-4bee-a224-ba38f06d2a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:34.048247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:52:34.085693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84915cd-9545-48a7-bb4b-c9733cc6bcac None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:34.096644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84915cd-9545-48a7-bb4b-c9733cc6bcac None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:34.100083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843a79c-d966-44f7-a103-8138e0d4e8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:34.100747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a79c-d966-44f7-a103-8138e0d4e8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:34.101030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a79c-d966-44f7-a103-8138e0d4e8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:34.101306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a79c-d966-44f7-a103-8138e0d4e8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:34.110657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843a79c-d966-44f7-a103-8138e0d4e8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:34.112064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843a79c-d966-44f7-a103-8138e0d4e8d5 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:34.122309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 720/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:36.469699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9feda-7f4a-46cf-ace5-ff071d043e87 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:36.470263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9feda-7f4a-46cf-ace5-ff071d043e87 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:36.470263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9feda-7f4a-46cf-ace5-ff071d043e87 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:36.471736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:36.477303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0e78-2ba5-4ef4-8878-18fc0a95855f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:36.477729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0e78-2ba5-4ef4-8878-18fc0a95855f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:36.477831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0e78-2ba5-4ef4-8878-18fc0a95855f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:36.483810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=70 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:52:36.497229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ef6f0e78-2ba5-4ef4-8878-18fc0a95855f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:36.498420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef6f0e78-2ba5-4ef4-8878-18fc0a95855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:36.506156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:52:36.544208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9feda-7f4a-46cf-ace5-ff071d043e87 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:36.549575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=73 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:52:36.556782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9feda-7f4a-46cf-ace5-ff071d043e87 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:36.560218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca6fb4-b47c-4158-af48-11e0f3f9cd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:36.560875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca6fb4-b47c-4158-af48-11e0f3f9cd71 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:36.561156 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca6fb4-b47c-4158-af48-11e0f3f9cd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:36.561425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca6fb4-b47c-4158-af48-11e0f3f9cd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:36.570152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca6fb4-b47c-4158-af48-11e0f3f9cd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:36.572549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca6fb4-b47c-4158-af48-11e0f3f9cd71 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:36.581686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 740/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:38.878743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe1a26-d7fc-4248-8842-27c45e521d2a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:38.879269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe1a26-d7fc-4248-8842-27c45e521d2a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:38.879269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe1a26-d7fc-4248-8842-27c45e521d2a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:38.880651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:38.887058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c999ac-3d19-4453-8fc3-96c752899ce7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:38.887500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c999ac-3d19-4453-8fc3-96c752899ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:38.887716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c999ac-3d19-4453-8fc3-96c752899ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:38.905173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-48c999ac-3d19-4453-8fc3-96c752899ce7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:38.906348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-48c999ac-3d19-4453-8fc3-96c752899ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:38.913978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:52:38.951730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe1a26-d7fc-4248-8842-27c45e521d2a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:38.962292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe1a26-d7fc-4248-8842-27c45e521d2a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:38.966070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218daef-f049-4f58-8a63-18267262fbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:38.966707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218daef-f049-4f58-8a63-18267262fbda service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:38.966991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218daef-f049-4f58-8a63-18267262fbda service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:38.967267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218daef-f049-4f58-8a63-18267262fbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:38.976228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218daef-f049-4f58-8a63-18267262fbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:38.977952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218daef-f049-4f58-8a63-18267262fbda service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:38.987602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 723/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:41.254553 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13353371-2bc6-4149-853b-fd8a681a5925 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:41.255107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13353371-2bc6-4149-853b-fd8a681a5925 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:41.255107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13353371-2bc6-4149-853b-fd8a681a5925 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:41.256549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:41.261921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19949c77-ae7c-49bb-9717-0a661af03b62 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:41.262334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19949c77-ae7c-49bb-9717-0a661af03b62 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:41.262470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19949c77-ae7c-49bb-9717-0a661af03b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:41.280820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-19949c77-ae7c-49bb-9717-0a661af03b62 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:41.282019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-19949c77-ae7c-49bb-9717-0a661af03b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:41.290551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:52:41.327595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13353371-2bc6-4149-853b-fd8a681a5925 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:41.339380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13353371-2bc6-4149-853b-fd8a681a5925 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:41.342885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cd997-c3bc-4066-aa22-31db746f1dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:41.343515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cd997-c3bc-4066-aa22-31db746f1dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:41.343795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cd997-c3bc-4066-aa22-31db746f1dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:41.344072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cd997-c3bc-4066-aa22-31db746f1dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:41.353284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cd997-c3bc-4066-aa22-31db746f1dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:41.354693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cd997-c3bc-4066-aa22-31db746f1dd9 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:41.363916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 743/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:43.640971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de12a7-c581-4986-9c0f-001c9d0e47cb None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:43.641523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de12a7-c581-4986-9c0f-001c9d0e47cb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:43.641523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de12a7-c581-4986-9c0f-001c9d0e47cb None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:43.642893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:52:43.649081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320b107-73da-4919-944c-8fdbac17f119 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:43.649775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320b107-73da-4919-944c-8fdbac17f119 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:43.649958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320b107-73da-4919-944c-8fdbac17f119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:43.667973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3320b107-73da-4919-944c-8fdbac17f119 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:52:43.669185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3320b107-73da-4919-944c-8fdbac17f119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:52:43.676856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:52:43.714148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de12a7-c581-4986-9c0f-001c9d0e47cb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:52:43.724721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de12a7-c581-4986-9c0f-001c9d0e47cb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:52:43.728512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda7be0-676a-4efa-986e-3b7827177ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:52:43.729140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7be0-676a-4efa-986e-3b7827177ffe service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:52:43.729423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7be0-676a-4efa-986e-3b7827177ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:52:43.729695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7be0-676a-4efa-986e-3b7827177ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:52:43.738379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda7be0-676a-4efa-986e-3b7827177ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:52:43.739799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda7be0-676a-4efa-986e-3b7827177ffe service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:52:43.749170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 726/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:52:53.662563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=17 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:52:53.745048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=17 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:53:15.508726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996503e-2748-451b-9fa4-14958c63cc1e None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:15.509324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996503e-2748-451b-9fa4-14958c63cc1e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:15.509324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996503e-2748-451b-9fa4-14958c63cc1e None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:15.510551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 745/1471] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:15.515855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3428d9-d9bb-4bb0-91d2-55e895bb2edd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:15.516259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3428d9-d9bb-4bb0-91d2-55e895bb2edd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:15.516435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3428d9-d9bb-4bb0-91d2-55e895bb2edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:15.534995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ee3428d9-d9bb-4bb0-91d2-55e895bb2edd None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:15.536220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ee3428d9-d9bb-4bb0-91d2-55e895bb2edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:15.544175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 727/1472] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:15.590746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75d78c-89fe-45f7-9975-9a40d5afc74f None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:15.591029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75d78c-89fe-45f7-9975-9a40d5afc74f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:15.591303 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75d78c-89fe-45f7-9975-9a40d5afc74f None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:15.592689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 746/1473] 10.222.0.18 () {54 vars in 962 bytes} [Tue Jun 7 15:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:15.597415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51561592-bf87-407b-9d89-be3fe7dc3671 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:15.597703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51561592-bf87-407b-9d89-be3fe7dc3671 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:15.597978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51561592-bf87-407b-9d89-be3fe7dc3671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:15.616463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-51561592-bf87-407b-9d89-be3fe7dc3671 None None] MFA Rules not processed for user `ac0206e09e4f426d9f4b54543d89c039`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:15.617646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-51561592-bf87-407b-9d89-be3fe7dc3671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:15.624084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 728/1474] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Jun 7 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:15.632498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75d78c-89fe-45f7-9975-9a40d5afc74f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:53:15.644934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75d78c-89fe-45f7-9975-9a40d5afc74f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:53:15.649760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad1e47-44c8-4e24-83e3-650c8b439959 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0206e09e4f426d9f4b54543d89c039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:53:15.650639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad1e47-44c8-4e24-83e3-650c8b439959 service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:15.651155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad1e47-44c8-4e24-83e3-650c8b439959 service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:15.651664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad1e47-44c8-4e24-83e3-650c8b439959 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:15.665064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad1e47-44c8-4e24-83e3-650c8b439959 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:53:15.666527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad1e47-44c8-4e24-83e3-650c8b439959 service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:53:15.676073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 747/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:53:18.031145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39b2b-00c8-489a-a64f-da1b1411bd55 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:18.031689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39b2b-00c8-489a-a64f-da1b1411bd55 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:18.031689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39b2b-00c8-489a-a64f-da1b1411bd55 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:18.033139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 729/1476] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:18.039224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db960b4-b5c1-400d-9c8b-51e84c1781cc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:18.039809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db960b4-b5c1-400d-9c8b-51e84c1781cc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:18.039809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db960b4-b5c1-400d-9c8b-51e84c1781cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:18.057223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0db960b4-b5c1-400d-9c8b-51e84c1781cc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:18.058386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0db960b4-b5c1-400d-9c8b-51e84c1781cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:18.065954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 748/1477] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:18.090416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39b2b-00c8-489a-a64f-da1b1411bd55 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:53:18.103091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39b2b-00c8-489a-a64f-da1b1411bd55 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:53:18.107441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcd336-fffa-4455-a7f2-bfc2e4764e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:53:18.108076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcd336-fffa-4455-a7f2-bfc2e4764e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:18.108371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcd336-fffa-4455-a7f2-bfc2e4764e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:18.108642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcd336-fffa-4455-a7f2-bfc2e4764e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:18.109354 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcd336-fffa-4455-a7f2-bfc2e4764e4c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:53:18.111076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-b2dcd336-fffa-4455-a7f2-bfc2e4764e4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:53:18.111076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:53:18.111076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:53:18.112196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcd336-fffa-4455-a7f2-bfc2e4764e4c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:53:18.132397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 730/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Jun 7 15:53:18 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:53:20.398275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5418fe-ebe6-45d4-a18e-65fda4432ed7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:20.398832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5418fe-ebe6-45d4-a18e-65fda4432ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:20.398832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5418fe-ebe6-45d4-a18e-65fda4432ed7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:20.400198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 749/1479] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:20.405342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b671cd-a21d-4e80-b647-c8c29bf98035 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:20.405764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b671cd-a21d-4e80-b647-c8c29bf98035 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:20.405884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b671cd-a21d-4e80-b647-c8c29bf98035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:20.423693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e1b671cd-a21d-4e80-b647-c8c29bf98035 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:20.424916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e1b671cd-a21d-4e80-b647-c8c29bf98035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:20.432825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 731/1480] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:20.477624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5418fe-ebe6-45d4-a18e-65fda4432ed7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:53:20.492095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5418fe-ebe6-45d4-a18e-65fda4432ed7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:53:20.497482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395023-eec3-40eb-8244-02fe3cdd3ff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:53:20.498109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395023-eec3-40eb-8244-02fe3cdd3ff1 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:20.498392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395023-eec3-40eb-8244-02fe3cdd3ff1 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:20.498667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395023-eec3-40eb-8244-02fe3cdd3ff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:20.507396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395023-eec3-40eb-8244-02fe3cdd3ff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:53:20.509002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395023-eec3-40eb-8244-02fe3cdd3ff1 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:53:20.518463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 750/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:53:22.980214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff86be-c220-4bf2-8496-fbe557cd58b8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:22.980821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff86be-c220-4bf2-8496-fbe557cd58b8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:22.980821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff86be-c220-4bf2-8496-fbe557cd58b8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:22.982227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 732/1482] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:22.987521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e145b3-109f-43d6-8559-1a89a0eccb31 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:22.987947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e145b3-109f-43d6-8559-1a89a0eccb31 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:22.988075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e145b3-109f-43d6-8559-1a89a0eccb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:23.005877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-13e145b3-109f-43d6-8559-1a89a0eccb31 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:23.007654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-13e145b3-109f-43d6-8559-1a89a0eccb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:23.015288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 751/1483] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:23.050708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff86be-c220-4bf2-8496-fbe557cd58b8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:53:23.061934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff86be-c220-4bf2-8496-fbe557cd58b8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:53:23.065375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be43ecf-dba2-4f3e-8a95-fd8536673ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:53:23.066001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be43ecf-dba2-4f3e-8a95-fd8536673ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:23.066280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be43ecf-dba2-4f3e-8a95-fd8536673ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:23.066548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be43ecf-dba2-4f3e-8a95-fd8536673ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:23.075939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be43ecf-dba2-4f3e-8a95-fd8536673ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:53:23.077381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be43ecf-dba2-4f3e-8a95-fd8536673ca6 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:53:23.086620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 733/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:53:25.410420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f61a6c-c9dd-4692-8697-b584e666c1d2 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:25.410949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f61a6c-c9dd-4692-8697-b584e666c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:25.410949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f61a6c-c9dd-4692-8697-b584e666c1d2 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:25.412289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 752/1485] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:25.418260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b756932-00af-4b8b-a344-bb9f6022fff0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:25.418684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b756932-00af-4b8b-a344-bb9f6022fff0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:25.418824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b756932-00af-4b8b-a344-bb9f6022fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:25.436863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9b756932-00af-4b8b-a344-bb9f6022fff0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:25.438046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9b756932-00af-4b8b-a344-bb9f6022fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:25.448076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 734/1486] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:25.487589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f61a6c-c9dd-4692-8697-b584e666c1d2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:53:25.499629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f61a6c-c9dd-4692-8697-b584e666c1d2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:53:25.503713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896ad01-0d5c-4641-abf6-b1fcda548c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:53:25.504374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ad01-0d5c-4641-abf6-b1fcda548c3b service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:25.504674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ad01-0d5c-4641-abf6-b1fcda548c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:25.504960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ad01-0d5c-4641-abf6-b1fcda548c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:25.517388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896ad01-0d5c-4641-abf6-b1fcda548c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:53:25.518812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896ad01-0d5c-4641-abf6-b1fcda548c3b service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:53:25.528705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 753/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:53:28.055893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a1d7-7554-4504-b134-e56fefd545d7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:28.056381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a1d7-7554-4504-b134-e56fefd545d7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:28.056494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a1d7-7554-4504-b134-e56fefd545d7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:28.057856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 735/1488] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:28.063109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3d82-84d9-4c6b-b00f-85ca274e157d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:28.063525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3d82-84d9-4c6b-b00f-85ca274e157d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:28.063734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3d82-84d9-4c6b-b00f-85ca274e157d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:28.080985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5a2d3d82-84d9-4c6b-b00f-85ca274e157d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:28.082168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5a2d3d82-84d9-4c6b-b00f-85ca274e157d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:28.089892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 754/1489] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:28.130895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a1d7-7554-4504-b134-e56fefd545d7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:53:28.141489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a1d7-7554-4504-b134-e56fefd545d7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:53:28.144952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e454b3-9a11-426a-8665-6b2c9e2da72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:53:28.145580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e454b3-9a11-426a-8665-6b2c9e2da72f service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:28.145859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e454b3-9a11-426a-8665-6b2c9e2da72f service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:28.146184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e454b3-9a11-426a-8665-6b2c9e2da72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:28.155729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e454b3-9a11-426a-8665-6b2c9e2da72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:53:28.157184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e454b3-9a11-426a-8665-6b2c9e2da72f service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:53:28.167142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 736/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:53:30.717588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f6166-15e6-4477-95ba-7f7fd3b25ea4 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:30.718202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f6166-15e6-4477-95ba-7f7fd3b25ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:30.718202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f6166-15e6-4477-95ba-7f7fd3b25ea4 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:30.719505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 755/1491] 10.222.0.18 () {54 vars in 926 bytes} [Tue Jun 7 15:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:53:30.724683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0b7e6-4c96-4213-a893-4db4bd2f2064 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:30.725933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0b7e6-4c96-4213-a893-4db4bd2f2064 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:30.726272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0b7e6-4c96-4213-a893-4db4bd2f2064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:30.744186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7de0b7e6-4c96-4213-a893-4db4bd2f2064 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:53:30.745393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7de0b7e6-4c96-4213-a893-4db4bd2f2064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:53:30.753006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 737/1492] 10.222.0.18 () {58 vars in 1023 bytes} [Tue Jun 7 15:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:53:30.792546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f6166-15e6-4477-95ba-7f7fd3b25ea4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:53:30.803048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f6166-15e6-4477-95ba-7f7fd3b25ea4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:53:30.806466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938c579-eee5-4412-8c3e-9ecfd37b530d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:53:30.807085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938c579-eee5-4412-8c3e-9ecfd37b530d service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:53:30.807361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938c579-eee5-4412-8c3e-9ecfd37b530d service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:53:30.807629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938c579-eee5-4412-8c3e-9ecfd37b530d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:53:30.816298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938c579-eee5-4412-8c3e-9ecfd37b530d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:53:30.817728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938c579-eee5-4412-8c3e-9ecfd37b530d service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:53:30.827535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 756/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:53:40.738743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=33 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:53:40.823466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=33 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:55:03.141426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa5f74-09bc-4ce0-ac44-5be6f907ffa4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.141993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa5f74-09bc-4ce0-ac44-5be6f907ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.141993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa5f74-09bc-4ce0-ac44-5be6f907ffa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.160858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-59aa5f74-09bc-4ce0-ac44-5be6f907ffa4 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:55:03.162070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-59aa5f74-09bc-4ce0-ac44-5be6f907ffa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:55:03.169956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 738/1494] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:55:03.178280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0039824-875e-4a97-8bb1-226aba2f9987 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.178756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0039824-875e-4a97-8bb1-226aba2f9987 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.178913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0039824-875e-4a97-8bb1-226aba2f9987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.197154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c0039824-875e-4a97-8bb1-226aba2f9987 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:55:03.198325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c0039824-875e-4a97-8bb1-226aba2f9987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:55:03.206085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 757/1495] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:55:03.211165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa5f74-09bc-4ce0-ac44-5be6f907ffa4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.220691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa5f74-09bc-4ce0-ac44-5be6f907ffa4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.225396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.226039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.226319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.226592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.227334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:55:03.227890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.228926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:03.228926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:55:03.228926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:55:03.229853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.241873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 739/1496] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:55:03.249970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0039824-875e-4a97-8bb1-226aba2f9987 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.259662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0039824-875e-4a97-8bb1-226aba2f9987 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.264247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.264920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.265202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.265478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.266172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.267904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:03.267904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Jun 07 15:55:03.267904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: Jun 07 15:55:03.269933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.304730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 758/1497] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:55:03 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:55:03.308781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.319268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1a80-8254-40b2-9f3d-664db5469a46 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.374364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.375027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.375309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.375584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.376271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.377303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:03.377303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:55:03.377303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:55:03.379056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.398616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 740/1498] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:55:03 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:55:03.402799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.412848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce829808-987b-4217-8e52-618e6fa40f95 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.417293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.417931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.418209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.418482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.419186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.420175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:03.420175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Jun 07 15:55:03.420175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: Jun 07 15:55:03.421080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.434282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 759/1499] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:55:03.438156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.448605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df162ee0-9df5-4d55-ac1c-13ee09083362 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.453142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fff05-03e6-44e2-b28c-c27d54821af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.453789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fff05-03e6-44e2-b28c-c27d54821af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.454071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fff05-03e6-44e2-b28c-c27d54821af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.454347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fff05-03e6-44e2-b28c-c27d54821af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.455063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fff05-03e6-44e2-b28c-c27d54821af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.456066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-ac2fff05-03e6-44e2-b28c-c27d54821af7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:03.456066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:55:03.456066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:55:03.457000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fff05-03e6-44e2-b28c-c27d54821af7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.470069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 741/1500] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:55:03.473969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.483972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fcb23-f32a-4603-b503-8a28aa94e8f3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.489069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457ff6e-1504-46f8-a603-f0331cfbafa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.489936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457ff6e-1504-46f8-a603-f0331cfbafa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.490215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457ff6e-1504-46f8-a603-f0331cfbafa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.490489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457ff6e-1504-46f8-a603-f0331cfbafa5 admin admin] PATH_INFO: `/v3/projects/b17a78b194f0478dab28a054afbd3c7e/users/b23b15309a8a41959864f97f3b16c1b9/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.505608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457ff6e-1504-46f8-a603-f0331cfbafa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17a78b194f0478dab28a054afbd3c7e, user_id=b23b15309a8a41959864f97f3b16c1b9, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.506622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-7457ff6e-1504-46f8-a603-f0331cfbafa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:03.506622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Jun 07 15:55:03.506622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: Jun 07 15:55:03.508575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457ff6e-1504-46f8-a603-f0331cfbafa5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.521702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 760/1501] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:55:03 2022] PUT /identity/v3/projects/b17a78b194f0478dab28a054afbd3c7e/users/b23b15309a8a41959864f97f3b16c1b9/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:55:03.534149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac82917-78d4-4915-baed-1384fbff2ecb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.534440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac82917-78d4-4915-baed-1384fbff2ecb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.534918 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac82917-78d4-4915-baed-1384fbff2ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.560871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fac82917-78d4-4915-baed-1384fbff2ecb None None] MFA Rules not processed for user `b23b15309a8a41959864f97f3b16c1b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:55:03.562064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fac82917-78d4-4915-baed-1384fbff2ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:55:03.625179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 742/1502] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:55:03.638873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f45402-6d2b-476d-aa12-b5aebee47beb None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.639174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f45402-6d2b-476d-aa12-b5aebee47beb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.639436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f45402-6d2b-476d-aa12-b5aebee47beb None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.640844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 761/1503] 10.222.0.18 () {54 vars in 963 bytes} [Tue Jun 7 15:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:55:03.646364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894a04-f35d-4e71-a761-f468a53e5ea6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.646654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894a04-f35d-4e71-a761-f468a53e5ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.646928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894a04-f35d-4e71-a761-f468a53e5ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.664801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-23894a04-f35d-4e71-a761-f468a53e5ea6 None None] MFA Rules not processed for user `a31ce1720fa84c43aec2ea016371f475`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:55:03.665912 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-23894a04-f35d-4e71-a761-f468a53e5ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:55:03.710305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:55:03.718860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 743/1504] 10.222.0.18 () {58 vars in 1060 bytes} [Tue Jun 7 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:55:03.726831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f45402-6d2b-476d-aa12-b5aebee47beb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.738188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f45402-6d2b-476d-aa12-b5aebee47beb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.741792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.742420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.742700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.742976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.751284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.752731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.762174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 762/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:55:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:55:03.813118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220f91-2a31-4d71-b205-b7f462a61a10 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.813625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220f91-2a31-4d71-b205-b7f462a61a10 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.813625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220f91-2a31-4d71-b205-b7f462a61a10 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.814893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 744/1506] 10.222.0.18 () {52 vars in 829 bytes} [Tue Jun 7 15:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:55:03.819396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:03.883671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cc657-467c-41b3-8180-180383431f38 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:03.888121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:03.889014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:03.889299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:03.889570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] PATH_INFO: `/v3/users/b23b15309a8a41959864f97f3b16c1b9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:03.890732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23b15309a8a41959864f97f3b16c1b9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:03.891728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:03.891728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Jun 07 15:55:03.891728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: Jun 07 15:55:03.892979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:03.961506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23b15309a8a41959864f97f3b16c1b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:03.962172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23b15309a8a41959864f97f3b16c1b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:03.962784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23b15309a8a41959864f97f3b16c1b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:03.975859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23b15309a8a41959864f97f3b16c1b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:03.995312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23b15309a8a41959864f97f3b16c1b9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:03.999245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:55:04.013218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 763/1507] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:55:03 2022] DELETE /identity/v3/users/b23b15309a8a41959864f97f3b16c1b9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:55:04.017693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:04.017972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:04.018241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:04.036835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:55:04.038082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:55:04.098762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 745/1508] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:55:04.109058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:04.110401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:55:04.174137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79a537-44ee-4619-a631-2fa7d880fef3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:04.179397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4c6a7-60b5-48a8-ab13-9e6976bf30f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:04.180339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c6a7-60b5-48a8-ab13-9e6976bf30f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:04.180655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c6a7-60b5-48a8-ab13-9e6976bf30f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:04.180927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c6a7-60b5-48a8-ab13-9e6976bf30f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:04.191360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c6a7-60b5-48a8-ab13-9e6976bf30f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:04.192842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c6a7-60b5-48a8-ab13-9e6976bf30f3 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:04.202283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 764/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:55:04.532135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:55:04.533719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:55:04.546459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329f8c2-eda9-491b-9877-23ceace331f1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:55:04.551598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:55:04.552521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:55:04.552801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:55:04.553074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] PATH_INFO: `/v3/projects/b17a78b194f0478dab28a054afbd3c7e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:55:04.554319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17a78b194f0478dab28a054afbd3c7e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:55:04.555299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:55:04.555299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:55:04.555299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:55:04.556608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:55:04.617362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17a78b194f0478dab28a054afbd3c7e', 'request_id': 'req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:04.631169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17a78b194f0478dab28a054afbd3c7e', 'request_id': 'req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:04.640862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17a78b194f0478dab28a054afbd3c7e', 'request_id': 'req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:04.652627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] The token cache is being invalidate because project b17a78b194f0478dab28a054afbd3c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:55:04.652918 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:04.653637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e7f212ad-cd45-447a-a36d-0c27d9a78fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:55:04.655199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 746/1510] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:55:04 2022] DELETE /identity/v3/projects/b17a78b194f0478dab28a054afbd3c7e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:55:14.198039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:55:14.649661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:56:40.924229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b534e-ea23-429d-b11c-8cdbbcca7b86 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:40.924859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b534e-ea23-429d-b11c-8cdbbcca7b86 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:40.924859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b534e-ea23-429d-b11c-8cdbbcca7b86 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:40.926220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:56:40.930965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8320b-5d8a-4e36-8515-2f664bfa5603 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:40.931423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8320b-5d8a-4e36-8515-2f664bfa5603 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:40.931530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8320b-5d8a-4e36-8515-2f664bfa5603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:40.949888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9cf8320b-5d8a-4e36-8515-2f664bfa5603 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:56:40.951075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9cf8320b-5d8a-4e36-8515-2f664bfa5603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:41.009523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:56:41.201471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b534e-ea23-429d-b11c-8cdbbcca7b86 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:56:41.204230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8a1b534e-ea23-429d-b11c-8cdbbcca7b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:41.282826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b534e-ea23-429d-b11c-8cdbbcca7b86 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:56:41.288658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23ba9d-e297-46d5-86af-246e0591844a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:56:41.289311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ba9d-e297-46d5-86af-246e0591844a service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:41.289593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ba9d-e297-46d5-86af-246e0591844a service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:41.289869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ba9d-e297-46d5-86af-246e0591844a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:41.299481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23ba9d-e297-46d5-86af-246e0591844a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:56:41.301151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23ba9d-e297-46d5-86af-246e0591844a service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:56:41.310889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 766/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:56:42.216230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ee4ce-9cbe-41bf-8a62-8d98e5e079c7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:42.216768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ee4ce-9cbe-41bf-8a62-8d98e5e079c7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:42.216904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ee4ce-9cbe-41bf-8a62-8d98e5e079c7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:42.218787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:56:42.223658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b135985-c70d-4e69-80a0-f7818399dbd1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:42.224095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b135985-c70d-4e69-80a0-f7818399dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:42.224305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b135985-c70d-4e69-80a0-f7818399dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:42.242779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0b135985-c70d-4e69-80a0-f7818399dbd1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:56:42.243974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0b135985-c70d-4e69-80a0-f7818399dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:42.251848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:56:42.407581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ee4ce-9cbe-41bf-8a62-8d98e5e079c7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:56:42.413248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d39a-bf8e-4d98-9fae-cf03c0ceb483 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:42.413596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d39a-bf8e-4d98-9fae-cf03c0ceb483 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:42.413873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d39a-bf8e-4d98-9fae-cf03c0ceb483 None None] PATH_INFO: `/v3` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:42.415219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 768/1516] 10.222.0.24 () {54 vars in 925 bytes} [Tue Jun 7 15:56:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:56:42.422300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ee4ce-9cbe-41bf-8a62-8d98e5e079c7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:56:42.427620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04b254-802f-46f4-af00-87a45d8d1593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:56:42.428251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04b254-802f-46f4-af00-87a45d8d1593 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:42.428561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04b254-802f-46f4-af00-87a45d8d1593 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:42.428841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04b254-802f-46f4-af00-87a45d8d1593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:42.437199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04b254-802f-46f4-af00-87a45d8d1593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:56:42.438782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04b254-802f-46f4-af00-87a45d8d1593 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:56:42.448292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 749/1517] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:56:42.449159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1d2f9-ab92-4d9e-94eb-2593b39a90e5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:42.449513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1d2f9-ab92-4d9e-94eb-2593b39a90e5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:42.449794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1d2f9-ab92-4d9e-94eb-2593b39a90e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:42.468703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-81e1d2f9-ab92-4d9e-94eb-2593b39a90e5 None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:56:42.469953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-81e1d2f9-ab92-4d9e-94eb-2593b39a90e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:42.538492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 769/1518] 10.222.0.24 () {58 vars in 1015 bytes} [Tue Jun 7 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:56:46.586055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975cd58-49e6-4edb-819f-652be7bea6e5 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:46.586617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975cd58-49e6-4edb-819f-652be7bea6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:46.586617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975cd58-49e6-4edb-819f-652be7bea6e5 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:46.588018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 750/1519] 10.222.0.18 () {54 vars in 960 bytes} [Tue Jun 7 15:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:56:46.593472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5c386-7e12-425d-af04-0a4827e61fdc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:46.593882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5c386-7e12-425d-af04-0a4827e61fdc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:46.594003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5c386-7e12-425d-af04-0a4827e61fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:46.612803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-12b5c386-7e12-425d-af04-0a4827e61fdc None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:56:46.613963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-12b5c386-7e12-425d-af04-0a4827e61fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:46.622326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 770/1520] 10.222.0.18 () {58 vars in 1057 bytes} [Tue Jun 7 15:56:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:56:46.630379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975cd58-49e6-4edb-819f-652be7bea6e5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:56:46.644433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975cd58-49e6-4edb-819f-652be7bea6e5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:56:46.649419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e71264-d265-48af-948b-7b756efa9055 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:56:46.650059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e71264-d265-48af-948b-7b756efa9055 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:46.650346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e71264-d265-48af-948b-7b756efa9055 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:46.650631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e71264-d265-48af-948b-7b756efa9055 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:46.659201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e71264-d265-48af-948b-7b756efa9055 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:56:46.660812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e71264-d265-48af-948b-7b756efa9055 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:56:46.670945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 751/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:56:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:56:53.751152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310b779-201e-4814-a691-2e359357f55b None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:53.751729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310b779-201e-4814-a691-2e359357f55b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:53.751729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310b779-201e-4814-a691-2e359357f55b None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:53.753122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:56:53.757773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaacb0-e334-4141-80ef-f09cd76b48d9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:53.758175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaacb0-e334-4141-80ef-f09cd76b48d9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:53.758328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaacb0-e334-4141-80ef-f09cd76b48d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:53.777235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6ddaacb0-e334-4141-80ef-f09cd76b48d9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:56:53.778459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6ddaacb0-e334-4141-80ef-f09cd76b48d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:53.787353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:56:53.942313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310b779-201e-4814-a691-2e359357f55b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:56:53.957581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310b779-201e-4814-a691-2e359357f55b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:56:53.962914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aac268-17b5-4460-aa4d-8fa9911c8bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:56:53.963545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aac268-17b5-4460-aa4d-8fa9911c8bd4 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:53.963831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aac268-17b5-4460-aa4d-8fa9911c8bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:53.964109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aac268-17b5-4460-aa4d-8fa9911c8bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:53.972319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aac268-17b5-4460-aa4d-8fa9911c8bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:56:53.973914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aac268-17b5-4460-aa4d-8fa9911c8bd4 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:56:53.983586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 772/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:56:54.845915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ec68-6521-4334-9a27-3f1a8c4df5fb None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:54.846488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ec68-6521-4334-9a27-3f1a8c4df5fb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:54.846488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ec68-6521-4334-9a27-3f1a8c4df5fb None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:54.847842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 15:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:56:54.852505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33fcbc-1598-4d7c-9a8d-f179a7249775 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:54.852950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33fcbc-1598-4d7c-9a8d-f179a7249775 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:54.853057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33fcbc-1598-4d7c-9a8d-f179a7249775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:54.873300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0f33fcbc-1598-4d7c-9a8d-f179a7249775 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:56:54.874498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0f33fcbc-1598-4d7c-9a8d-f179a7249775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:54.882179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:56:55.035184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ec68-6521-4334-9a27-3f1a8c4df5fb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:56:55.049346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ec68-6521-4334-9a27-3f1a8c4df5fb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:56:55.054605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86ec88-6852-44b4-b69d-a243f0d08ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:56:55.055226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86ec88-6852-44b4-b69d-a243f0d08ab5 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:55.055505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86ec88-6852-44b4-b69d-a243f0d08ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:55.055779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86ec88-6852-44b4-b69d-a243f0d08ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:55.063737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86ec88-6852-44b4-b69d-a243f0d08ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:56:55.065330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86ec88-6852-44b4-b69d-a243f0d08ab5 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:56:55.074497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 754/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:56:58.978853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2888-b0af-44c4-96ed-6faafd0c5d9a None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:58.979426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2888-b0af-44c4-96ed-6faafd0c5d9a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:58.979426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2888-b0af-44c4-96ed-6faafd0c5d9a None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:58.980822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:56:58.986563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311b387-a7ff-4e98-b48d-60ec34306893 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:58.986981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311b387-a7ff-4e98-b48d-60ec34306893 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:58.987139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311b387-a7ff-4e98-b48d-60ec34306893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:59.005550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a311b387-a7ff-4e98-b48d-60ec34306893 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:56:59.006748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a311b387-a7ff-4e98-b48d-60ec34306893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:59.014823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:56:59.052040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d2888-b0af-44c4-96ed-6faafd0c5d9a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:56:59.053713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-370d2888-b0af-44c4-96ed-6faafd0c5d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:56:59.106334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:56:59.111570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d2888-b0af-44c4-96ed-6faafd0c5d9a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:56:59.116001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85458768-e1bd-4239-9ddc-123edfc81e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:56:59.116798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85458768-e1bd-4239-9ddc-123edfc81e6a service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:56:59.117327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85458768-e1bd-4239-9ddc-123edfc81e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:56:59.117825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85458768-e1bd-4239-9ddc-123edfc81e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:56:59.130741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85458768-e1bd-4239-9ddc-123edfc81e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:56:59.132186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85458768-e1bd-4239-9ddc-123edfc81e6a service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:56:59.142145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 775/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:57:01.369077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76311b-a55f-4c83-9326-f5ec4245ba3d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:01.369611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76311b-a55f-4c83-9326-f5ec4245ba3d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:01.369611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76311b-a55f-4c83-9326-f5ec4245ba3d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:01.371019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:57:01.376864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463e2f3-4fd9-4770-98ea-c7f554b08308 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:01.377272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463e2f3-4fd9-4770-98ea-c7f554b08308 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:01.377420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463e2f3-4fd9-4770-98ea-c7f554b08308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:01.395833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7463e2f3-4fd9-4770-98ea-c7f554b08308 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:57:01.397044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7463e2f3-4fd9-4770-98ea-c7f554b08308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:57:01.404768 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:57:01.441476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76311b-a55f-4c83-9326-f5ec4245ba3d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:57:01.443181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7a76311b-a55f-4c83-9326-f5ec4245ba3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:57:01.456611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76311b-a55f-4c83-9326-f5ec4245ba3d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:57:01.460079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a4957-5d77-42d9-b711-9391524c326c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:57:01.460748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a4957-5d77-42d9-b711-9391524c326c service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:01.461032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a4957-5d77-42d9-b711-9391524c326c service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:01.461308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a4957-5d77-42d9-b711-9391524c326c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:01.470057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a4957-5d77-42d9-b711-9391524c326c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:57:01.471469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a4957-5d77-42d9-b711-9391524c326c service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:57:01.481096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 757/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:57:05.284800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272a3ad-b70c-42c8-91bf-a8c903bc6fa9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:05.285396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272a3ad-b70c-42c8-91bf-a8c903bc6fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:05.285505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272a3ad-b70c-42c8-91bf-a8c903bc6fa9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:05.286945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:57:05.292073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ff8c5-e36b-4e67-8f04-5c33d6dc5ae8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:05.292543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ff8c5-e36b-4e67-8f04-5c33d6dc5ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:05.292647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ff8c5-e36b-4e67-8f04-5c33d6dc5ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:05.299755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=58 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:57:05.312143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-477ff8c5-e36b-4e67-8f04-5c33d6dc5ae8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:57:05.313433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-477ff8c5-e36b-4e67-8f04-5c33d6dc5ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:57:05.321240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:57:05.600127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272a3ad-b70c-42c8-91bf-a8c903bc6fa9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:57:05.601889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2272a3ad-b70c-42c8-91bf-a8c903bc6fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:57:05.659503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272a3ad-b70c-42c8-91bf-a8c903bc6fa9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:57:05.662990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0eec55-edfc-4edb-ae19-3b791ec87314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:57:05.663624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eec55-edfc-4edb-ae19-3b791ec87314 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:05.663903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eec55-edfc-4edb-ae19-3b791ec87314 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:05.664171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eec55-edfc-4edb-ae19-3b791ec87314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:05.673348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0eec55-edfc-4edb-ae19-3b791ec87314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:57:05.674757 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0eec55-edfc-4edb-ae19-3b791ec87314 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:57:05.683950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 778/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:57:07.967409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad88b41-f38b-4dd5-a326-a8f977b95ba6 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:07.967994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad88b41-f38b-4dd5-a326-a8f977b95ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:07.967994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad88b41-f38b-4dd5-a326-a8f977b95ba6 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:07.969924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 15:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:57:07.975102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9bf0-ff19-44d8-a7b9-a7916ba2756e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:07.975558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9bf0-ff19-44d8-a7b9-a7916ba2756e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:07.975680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9bf0-ff19-44d8-a7b9-a7916ba2756e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:07.994262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-734e9bf0-ff19-44d8-a7b9-a7916ba2756e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:57:07.995450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-734e9bf0-ff19-44d8-a7b9-a7916ba2756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:57:08.003723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 15:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:57:08.272742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad88b41-f38b-4dd5-a326-a8f977b95ba6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:57:08.274419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cad88b41-f38b-4dd5-a326-a8f977b95ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:57:08.287295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad88b41-f38b-4dd5-a326-a8f977b95ba6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:57:08.290784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cac46d-eecc-4f3c-8a52-4947732d4c9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:57:08.291416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac46d-eecc-4f3c-8a52-4947732d4c9c service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:57:08.291696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac46d-eecc-4f3c-8a52-4947732d4c9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:57:08.291972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac46d-eecc-4f3c-8a52-4947732d4c9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:57:08.301027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cac46d-eecc-4f3c-8a52-4947732d4c9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:57:08.302431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cac46d-eecc-4f3c-8a52-4947732d4c9c service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:57:08.311687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 760/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:57:17.988969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=36 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:57:18.307668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=8 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:39.871986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98daa00-9927-47a0-b064-ff1479194994 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:39.872664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98daa00-9927-47a0-b064-ff1479194994 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:39.872664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98daa00-9927-47a0-b064-ff1479194994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:39.881520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9b198-a129-4a71-9899-27c0bba9766a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:39.882462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9b198-a129-4a71-9899-27c0bba9766a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:39.882966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9b198-a129-4a71-9899-27c0bba9766a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:39.919827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b98daa00-9927-47a0-b064-ff1479194994 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:39.921062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b98daa00-9927-47a0-b064-ff1479194994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:39.926268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-42c9b198-a129-4a71-9899-27c0bba9766a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:39.927426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-42c9b198-a129-4a71-9899-27c0bba9766a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:39.943890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 780/1540] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:39.946724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 761/1541] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:58:39.953914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f90f5-2243-42b4-adf3-0b4a6fa8152b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:39.954208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f90f5-2243-42b4-adf3-0b4a6fa8152b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:39.954474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f90f5-2243-42b4-adf3-0b4a6fa8152b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:39.957879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633541e-ec77-46e9-b7bf-0e287aba1a1a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:39.958160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633541e-ec77-46e9-b7bf-0e287aba1a1a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:39.958431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633541e-ec77-46e9-b7bf-0e287aba1a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:39.995547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cc6f90f5-2243-42b4-adf3-0b4a6fa8152b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:39.996775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cc6f90f5-2243-42b4-adf3-0b4a6fa8152b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.006363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a633541e-ec77-46e9-b7bf-0e287aba1a1a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.007133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a633541e-ec77-46e9-b7bf-0e287aba1a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.017764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 781/1542] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.024804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe440a9-37fc-4806-8dfe-da0f83bf7dd5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.025113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe440a9-37fc-4806-8dfe-da0f83bf7dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.025393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe440a9-37fc-4806-8dfe-da0f83bf7dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.028297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 762/1543] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.029719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6f5e0-48c9-4816-bf3f-4e76ed3f344e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.030000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6f5e0-48c9-4816-bf3f-4e76ed3f344e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.030276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6f5e0-48c9-4816-bf3f-4e76ed3f344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.046476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bfe440a9-37fc-4806-8dfe-da0f83bf7dd5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.047661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bfe440a9-37fc-4806-8dfe-da0f83bf7dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.051248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f9c6f5e0-48c9-4816-bf3f-4e76ed3f344e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.052532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f9c6f5e0-48c9-4816-bf3f-4e76ed3f344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.056915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 782/1544] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.058265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06e48a-353a-4f2b-b3c9-04ba2b76a12a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.058548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06e48a-353a-4f2b-b3c9-04ba2b76a12a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.058821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06e48a-353a-4f2b-b3c9-04ba2b76a12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.060861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 763/1545] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.061657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6f5e0-48c9-4816-bf3f-4e76ed3f344e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.078942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6f5e0-48c9-4816-bf3f-4e76ed3f344e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.079813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-be06e48a-353a-4f2b-b3c9-04ba2b76a12a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.081148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-be06e48a-353a-4f2b-b3c9-04ba2b76a12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.084640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96580d9-cf7c-417b-b992-310afe8b5974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.085503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96580d9-cf7c-417b-b992-310afe8b5974 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.085784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96580d9-cf7c-417b-b992-310afe8b5974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.086168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96580d9-cf7c-417b-b992-310afe8b5974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.086894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96580d9-cf7c-417b-b992-310afe8b5974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.087297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96580d9-cf7c-417b-b992-310afe8b5974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.089130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96580d9-cf7c-417b-b992-310afe8b5974 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.090123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 783/1546] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.091075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06e48a-353a-4f2b-b3c9-04ba2b76a12a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.102075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06e48a-353a-4f2b-b3c9-04ba2b76a12a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.104974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 764/1547] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.106277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b2eac-4873-4749-b8f8-fc3e04106f3e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.106559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b2eac-4873-4749-b8f8-fc3e04106f3e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.106841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b2eac-4873-4749-b8f8-fc3e04106f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.115860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.116687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.116968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.117246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.118074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.118380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.119506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:58:40.119506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Jun 07 15:58:40.119506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: Jun 07 15:58:40.120404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83253df7-fc0a-4bb4-8c51-8d8c5676688a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.135726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 784/1548] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.137075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd2cce-c7de-4351-ab64-d95a066f394c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.137370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd2cce-c7de-4351-ab64-d95a066f394c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.137634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd2cce-c7de-4351-ab64-d95a066f394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.139319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5c6b2eac-4873-4749-b8f8-fc3e04106f3e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.141318 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5c6b2eac-4873-4749-b8f8-fc3e04106f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.156193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 765/1549] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.157544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b88cec-f031-44a4-b1bd-da9fe03f764a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.157830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b88cec-f031-44a4-b1bd-da9fe03f764a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.158041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f0bd2cce-c7de-4351-ab64-d95a066f394c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.158703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b88cec-f031-44a4-b1bd-da9fe03f764a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.159470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f0bd2cce-c7de-4351-ab64-d95a066f394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.168475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 785/1550] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.169742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c5e1-0a94-4534-a748-2dce52b4ce9d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.170021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c5e1-0a94-4534-a748-2dce52b4ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.170293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c5e1-0a94-4534-a748-2dce52b4ce9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.179225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-28b88cec-f031-44a4-b1bd-da9fe03f764a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.180451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-28b88cec-f031-44a4-b1bd-da9fe03f764a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.188955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 766/1551] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.189087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-16b5c5e1-0a94-4534-a748-2dce52b4ce9d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.189833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b88cec-f031-44a4-b1bd-da9fe03f764a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.190134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-16b5c5e1-0a94-4534-a748-2dce52b4ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.199066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 786/1552] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.199838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5c5e1-0a94-4534-a748-2dce52b4ce9d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.200308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b88cec-f031-44a4-b1bd-da9fe03f764a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.205139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bb753-aa09-456b-b68a-adc90285f886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.205810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb753-aa09-456b-b68a-adc90285f886 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.206654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb753-aa09-456b-b68a-adc90285f886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.206926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb753-aa09-456b-b68a-adc90285f886 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.207607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bb753-aa09-456b-b68a-adc90285f886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.208653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-731bb753-aa09-456b-b68a-adc90285f886 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:58:40.208653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:58:40.208653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:58:40.209592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5c5e1-0a94-4534-a748-2dce52b4ce9d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.209826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bb753-aa09-456b-b68a-adc90285f886 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.214102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79a215-faf4-4f9f-aa79-83e53469bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.214733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a215-faf4-4f9f-aa79-83e53469bed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.215013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a215-faf4-4f9f-aa79-83e53469bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.215285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a215-faf4-4f9f-aa79-83e53469bed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.215958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79a215-faf4-4f9f-aa79-83e53469bed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.218204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79a215-faf4-4f9f-aa79-83e53469bed2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.235694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 767/1553] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.237410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b968d96-fd5c-45f8-80ac-f6dc643401b7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.237704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b968d96-fd5c-45f8-80ac-f6dc643401b7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.238162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b968d96-fd5c-45f8-80ac-f6dc643401b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.246583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 787/1554] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.247867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c694-f05d-4188-98fb-28ecad8cd28e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.248159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c694-f05d-4188-98fb-28ecad8cd28e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.248567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c694-f05d-4188-98fb-28ecad8cd28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.256708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3b968d96-fd5c-45f8-80ac-f6dc643401b7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.257899 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3b968d96-fd5c-45f8-80ac-f6dc643401b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.266537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8880c694-f05d-4188-98fb-28ecad8cd28e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.267703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8880c694-f05d-4188-98fb-28ecad8cd28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.320540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 768/1555] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.321462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b968d96-fd5c-45f8-80ac-f6dc643401b7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.328274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 788/1556] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.329592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d09386-6fa7-40d2-af50-b01ba5525882 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.329890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d09386-6fa7-40d2-af50-b01ba5525882 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.330163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d09386-6fa7-40d2-af50-b01ba5525882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.331682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b968d96-fd5c-45f8-80ac-f6dc643401b7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.336747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.337406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.337686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.338176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.338904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.339207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.340839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.349207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c8d09386-6fa7-40d2-af50-b01ba5525882 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.350402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c8d09386-6fa7-40d2-af50-b01ba5525882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.352846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 769/1557] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.353658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.363809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 789/1558] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.363809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d09386-6fa7-40d2-af50-b01ba5525882 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.369557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d09386-6fa7-40d2-af50-b01ba5525882 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.369805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8703c5-a94d-4569-85aa-7fb803854d24 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.374058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.374422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.374698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.374977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.375102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.375248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.375352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.375605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.375952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.376253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.376398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.376648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.378411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.378761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.390018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 790/1559] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.390784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.391010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 770/1560] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.391781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.401557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318a25-d8b7-4024-a817-3bf8f7697a1a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.402908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10994075-2f2a-4c1a-b197-714dae7f20c2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.406423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839f702-de00-45ed-b476-3f25e408852f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.407101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f702-de00-45ed-b476-3f25e408852f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.407526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f702-de00-45ed-b476-3f25e408852f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.407638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.407803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f702-de00-45ed-b476-3f25e408852f admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.408207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.408512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.408622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839f702-de00-45ed-b476-3f25e408852f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.408787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.409496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.410001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-c839f702-de00-45ed-b476-3f25e408852f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:58:40.410001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Jun 07 15:58:40.410001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: Jun 07 15:58:40.411223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839f702-de00-45ed-b476-3f25e408852f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.411443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.437520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 791/1561] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.437725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 771/1562] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.437831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a870a6-bbaf-422b-9a8c-3e967eca75cd None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.437831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a870a6-bbaf-422b-9a8c-3e967eca75cd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.437831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a870a6-bbaf-422b-9a8c-3e967eca75cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.438052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.451067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f5de5-ea1d-4f7e-8217-8eb59e028daa admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.457216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38621aa1-e8d3-43c2-a493-419a334a9cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.457527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c3a870a6-bbaf-422b-9a8c-3e967eca75cd None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.457680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38621aa1-e8d3-43c2-a493-419a334a9cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.458159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38621aa1-e8d3-43c2-a493-419a334a9cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.458596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38621aa1-e8d3-43c2-a493-419a334a9cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.458746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c3a870a6-bbaf-422b-9a8c-3e967eca75cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.459328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38621aa1-e8d3-43c2-a493-419a334a9cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.459636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38621aa1-e8d3-43c2-a493-419a334a9cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.461432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38621aa1-e8d3-43c2-a493-419a334a9cd3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.466714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 792/1563] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.470646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a870a6-bbaf-422b-9a8c-3e967eca75cd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.481778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 772/1564] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.481778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ebb4-d7ea-42f3-9efe-0640ec63a5bc None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.481778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ebb4-d7ea-42f3-9efe-0640ec63a5bc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.481778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ebb4-d7ea-42f3-9efe-0640ec63a5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.482225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a870a6-bbaf-422b-9a8c-3e967eca75cd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.484877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.485520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.485798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.486067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.486746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.489789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.499127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0861ebb4-d7ea-42f3-9efe-0640ec63a5bc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:40.500446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0861ebb4-d7ea-42f3-9efe-0640ec63a5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:40.509385 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 773/1565] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:40.510019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ebb4-d7ea-42f3-9efe-0640ec63a5bc None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.519989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ebb4-d7ea-42f3-9efe-0640ec63a5bc None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.521681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 793/1566] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.522648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.546146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:40.553718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:40.587462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.588118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.588424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.588699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.589387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.590090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece10bfc-3f61-4dc3-8151-3b9312850613 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.591310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.594819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.595462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.595738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.596010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.596770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.598848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.617754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 774/1567] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.618608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.632735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 794/1568] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.632735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.684651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e9af9-76a8-408f-9aa9-232425905ebd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.689930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6d09d-413e-4c1f-a2d4-5da08766a94d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.690120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.691428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.691724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.692060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.692954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.694736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.695216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.695891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.696176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.696495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.697221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.698801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.705441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 775/1569] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:40.712132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 795/1570] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:40.712313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.712456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.719075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef994e-9714-4a6a-895b-917ca3d5a4b5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.719382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212f551-90f5-4624-b63a-ae91c96708ec admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.723810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.724141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.724478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.724766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.724899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.725039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.725136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.725321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.725753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.725993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.726112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.727620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.727865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.740075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 776/1571] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.740990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.750751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b4089-65c2-458c-80d4-3cbfb1268790 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.756130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.759632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 796/1572] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.759757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.759757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.759956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.760066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.760066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:40.760066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.760317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.773574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 777/1573] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:40.774187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.788578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64014066-0c8c-4988-85a6-852cd7b6c256 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.823738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be31624-cd6d-4cfc-a9d7-f391eb6cff52 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.828341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.829031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.829310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.829583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.830301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.832931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.847850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.848511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.848862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.849232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.849982 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.852062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.852613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 797/1574] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.853294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.863286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0a459-0fcc-4563-85a7-e0423e804b1f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.868434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.869079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.869360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.869627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.870300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.874955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 778/1575] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.875122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.875581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.880973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:40.883361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a4728-178f-454f-9253-346ee483da58 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.888894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.889663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.889948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.890223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.891068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.892714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.895107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 798/1576] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.895906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.903789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 779/1577] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:40.904611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.906159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c64688-ff4b-4299-b09f-be1da83ac242 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.909515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:40.911652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.912465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.912747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.913017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.913736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.914414 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354371b-b225-411c-a8bd-840f96bd5e40 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.915492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.921895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.922567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.922846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.923119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.923872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.926447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.927222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 799/1578] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:40.930473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.937515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc120e0-2f3d-4a84-bffd-0a1e17e410c3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:40.943557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:40.944831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:40.945251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:40.945738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:40.947003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:40.950088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:40.957713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 780/1579] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.957713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.974466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 800/1580] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:40.974466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:40.982333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b7a8-ecf2-4eeb-9f0a-a70e23d71b3a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.022393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f8b45-d7cd-4f43-b1cc-9bf38933cfdb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.027559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.028366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.028662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.028965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.029663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.031637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.040121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.040883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.041159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.041430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.042145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.043748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.054950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 801/1581] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.055769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.059559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 781/1582] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:40 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:41.060589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.065373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cfc47-0862-4c14-be74-593fcd1d29b5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.122707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.123384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.123658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.123930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.124755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.126571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.128765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e174a0c-baa7-4d9b-b306-11a799849325 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.135330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.136016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.136287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.136581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.137299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.138905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.142499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 802/1583] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.143107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.154035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 782/1584] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.154168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cb7ea-f09c-42a9-acd3-e1dfe80a8e51 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.154695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.159381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.160996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.161277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.161557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] PATH_INFO: `/v3/projects/9d0733c937754d1182c68bd4ea7f283c/users/8348af929ea740f3981631020a219f80/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.165161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec582df-ec08-4137-9122-784189048acf admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.169592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.170459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.170735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.171003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] PATH_INFO: `/v3/projects/fc49db8e3b744ef1972a6b8daff9550b/users/28ac0907283a4ba5be34011b2e8e83ec/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.178365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0733c937754d1182c68bd4ea7f283c, user_id=8348af929ea740f3981631020a219f80, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.181119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.186361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc49db8e3b744ef1972a6b8daff9550b, user_id=28ac0907283a4ba5be34011b2e8e83ec, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.187376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:58:41.187376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:58:41.187376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:58:41.189468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.194698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 803/1585] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:41 2022] PUT /identity/v3/projects/9d0733c937754d1182c68bd4ea7f283c/users/8348af929ea740f3981631020a219f80/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:41.195464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.205069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 783/1586] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:41 2022] PUT /identity/v3/projects/fc49db8e3b744ef1972a6b8daff9550b/users/28ac0907283a4ba5be34011b2e8e83ec/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:41.206397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.235801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:41.252338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:41.267245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee98d9-6779-4e43-baa0-a15b45c09c5a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.272533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.273201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.273476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.273893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.274575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.275616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef7b8-ffaa-4ce1-b010-511dcd75f483 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.276514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.280295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.280987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.281272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.281551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.282269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.283866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.294831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 784/1587] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.295636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.300894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 804/1588] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:41.300894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.306278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a61a6-dff6-424c-8eb6-4b56d93023b4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.309286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80431f77-a861-4269-a137-7dfdeda5e078 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.310979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.311628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.311906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.312177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.312925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.314050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.314535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.314868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.315223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.315502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.316190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.318155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.324645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 785/1589] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.325543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.335432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186a75-9d7d-42a4-b17e-e39a5051476b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.340149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e0ff4-bd5b-450e-b3d8-c1de7af26c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.340860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e0ff4-bd5b-450e-b3d8-c1de7af26c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.341142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e0ff4-bd5b-450e-b3d8-c1de7af26c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.341417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e0ff4-bd5b-450e-b3d8-c1de7af26c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.342242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e0ff4-bd5b-450e-b3d8-c1de7af26c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.346074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 805/1590] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:41.346074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.346339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e0ff4-bd5b-450e-b3d8-c1de7af26c1f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.353640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f4d92-5c55-40e0-b019-540a43ee527c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.359029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 786/1591] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.359029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294a06a-b0a7-449b-86f4-a2739bd3121a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.359029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294a06a-b0a7-449b-86f4-a2739bd3121a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.359029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294a06a-b0a7-449b-86f4-a2739bd3121a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.362049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d499a-51c2-4585-9a68-ad589a414075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.362693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d499a-51c2-4585-9a68-ad589a414075 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.362952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d499a-51c2-4585-9a68-ad589a414075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.363227 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d499a-51c2-4585-9a68-ad589a414075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.363969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d499a-51c2-4585-9a68-ad589a414075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.365608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d499a-51c2-4585-9a68-ad589a414075 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.375051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c294a06a-b0a7-449b-86f4-a2739bd3121a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:41.375911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 806/1592] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.376272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c294a06a-b0a7-449b-86f4-a2739bd3121a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:41.377260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce7d98-cedc-474f-909f-db8184ff4551 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.377559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce7d98-cedc-474f-909f-db8184ff4551 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.377991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce7d98-cedc-474f-909f-db8184ff4551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.384007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 787/1593] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:41.387105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294a06a-b0a7-449b-86f4-a2739bd3121a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.394649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294a06a-b0a7-449b-86f4-a2739bd3121a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.399170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.399823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.400105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.400405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.401123 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.402717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.405814 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-44ce7d98-cedc-474f-909f-db8184ff4551 None None] MFA Rules not processed for user `28ac0907283a4ba5be34011b2e8e83ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:41.407004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-44ce7d98-cedc-474f-909f-db8184ff4551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:41.413395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 788/1594] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.417198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.427216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d971a2e-603f-457d-92b0-6a184cb41b08 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.431713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.432453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.432753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.432977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.433869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.435546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.447112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 789/1595] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.447957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.459290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d014705-f64f-40a6-b05d-bb6d7ab5019c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.463857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.464751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.465055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.465324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] PATH_INFO: `/v3/projects/0e7218a263864d068e82a5845e1fa739/users/dfd3065e2d444dc6a4057713e738e0b4/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.473489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 807/1596] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:41.474351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce7d98-cedc-474f-909f-db8184ff4551 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.485594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce7d98-cedc-474f-909f-db8184ff4551 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.490256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7218a263864d068e82a5845e1fa739, user_id=dfd3065e2d444dc6a4057713e738e0b4, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.492536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.492953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.493186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.493468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.493740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.494465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.496510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.511483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 808/1597] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.511483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.511766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 790/1598] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:41 2022] PUT /identity/v3/projects/0e7218a263864d068e82a5845e1fa739/users/dfd3065e2d444dc6a4057713e738e0b4/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:41.511766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.522939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5818ac-f0e7-4623-927c-4a3e62394658 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.580138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6a0cb-6b58-43fa-bf66-b2e8b6a6a991 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.583527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.584445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.584729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.584950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.585085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] PATH_INFO: `/v3/projects/e601e41f827a4c6f8c743a174ac463fa/users/f47acc995d80476cbd4153e9121e97eb/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.585841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.586122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.586393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] PATH_INFO: `/v3/projects/a16c728ace3f4df5a25c28c3688dac6c/users/74408245319a4075b79d263326e19c84/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.599674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e601e41f827a4c6f8c743a174ac463fa, user_id=f47acc995d80476cbd4153e9121e97eb, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.602346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.602480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16c728ace3f4df5a25c28c3688dac6c, user_id=74408245319a4075b79d263326e19c84, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.604807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:41.605254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.617752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 809/1599] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:41 2022] PUT /identity/v3/projects/e601e41f827a4c6f8c743a174ac463fa/users/f47acc995d80476cbd4153e9121e97eb/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:41.618554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.625939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 791/1600] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:41 2022] PUT /identity/v3/projects/a16c728ace3f4df5a25c28c3688dac6c/users/74408245319a4075b79d263326e19c84/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:41.625939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.674268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:41.683226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79f8c-7e98-4544-9560-8a7d6de392c3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.693756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a4fa6-1455-4d8c-895c-4cc30d6bdb2b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.698879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.699708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.699990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.700270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] PATH_INFO: `/v3/projects/a59fb3181ede4a16876ce27ed77e6164/users/3c48ccd37a9d4b05952e87e9d3be16a4/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.718364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59fb3181ede4a16876ce27ed77e6164, user_id=3c48ccd37a9d4b05952e87e9d3be16a4, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.723757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.739531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 792/1601] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:41 2022] PUT /identity/v3/projects/a59fb3181ede4a16876ce27ed77e6164/users/3c48ccd37a9d4b05952e87e9d3be16a4/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:41.744548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.744685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf908d-6ac5-4ed6-8a49-ec994da1409d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.744685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf908d-6ac5-4ed6-8a49-ec994da1409d service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.744685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf908d-6ac5-4ed6-8a49-ec994da1409d service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.744685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf908d-6ac5-4ed6-8a49-ec994da1409d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.754907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf908d-6ac5-4ed6-8a49-ec994da1409d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.756960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf908d-6ac5-4ed6-8a49-ec994da1409d service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.767499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 810/1602] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:41.779058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd531e-9007-43a2-ac2d-cef60ac08ab7 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.779343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd531e-9007-43a2-ac2d-cef60ac08ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.779619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd531e-9007-43a2-ac2d-cef60ac08ab7 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.782427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 811/1603] 10.222.0.18 () {54 vars in 961 bytes} [Tue Jun 7 15:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:58:41.782427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd531e-9007-43a2-ac2d-cef60ac08ab7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:41.811183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ad73-d3fb-411c-b7d1-290773ea6db3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.816547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1db1f4-9119-4849-83d5-bc38bfcc1806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.817336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1db1f4-9119-4849-83d5-bc38bfcc1806 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.817721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1db1f4-9119-4849-83d5-bc38bfcc1806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.818180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1db1f4-9119-4849-83d5-bc38bfcc1806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.818972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1db1f4-9119-4849-83d5-bc38bfcc1806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.820755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1db1f4-9119-4849-83d5-bc38bfcc1806 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.834892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 793/1604] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.834892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ac83a-aa9f-46e9-90bc-f4544c8fe91c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.834892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ac83a-aa9f-46e9-90bc-f4544c8fe91c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.834892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ac83a-aa9f-46e9-90bc-f4544c8fe91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.864928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-333ac83a-aa9f-46e9-90bc-f4544c8fe91c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:41.866477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-333ac83a-aa9f-46e9-90bc-f4544c8fe91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:41.873757 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd531e-9007-43a2-ac2d-cef60ac08ab7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:41.877233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 794/1605] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:41.879156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe271e1-7cf1-47f9-aa0b-a4b8c917a6d3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.879495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe271e1-7cf1-47f9-aa0b-a4b8c917a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.879819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe271e1-7cf1-47f9-aa0b-a4b8c917a6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.883286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cb4ca-fd0b-4f6b-964a-483c48bdb38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:41.884167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cb4ca-fd0b-4f6b-964a-483c48bdb38d admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.884600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cb4ca-fd0b-4f6b-964a-483c48bdb38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.885602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cb4ca-fd0b-4f6b-964a-483c48bdb38d admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.885882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cb4ca-fd0b-4f6b-964a-483c48bdb38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:41.887710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cb4ca-fd0b-4f6b-964a-483c48bdb38d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:41.903088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 812/1606] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:41.903088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa09ce-37a8-4b69-9592-d1fa931549df None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.903088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa09ce-37a8-4b69-9592-d1fa931549df None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.903088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa09ce-37a8-4b69-9592-d1fa931549df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.908442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8fe271e1-7cf1-47f9-aa0b-a4b8c917a6d3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:41.910317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8fe271e1-7cf1-47f9-aa0b-a4b8c917a6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:41.922287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 795/1607] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:41.924551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d685-2f2d-48a0-bf5f-8a2b5624db39 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:41.924949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d685-2f2d-48a0-bf5f-8a2b5624db39 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:41.932457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-28aa09ce-37a8-4b69-9592-d1fa931549df None None] MFA Rules not processed for user `74408245319a4075b79d263326e19c84`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:41.932802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d685-2f2d-48a0-bf5f-8a2b5624db39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:41.933791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-28aa09ce-37a8-4b69-9592-d1fa931549df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:41.967497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5f77d685-2f2d-48a0-bf5f-8a2b5624db39 None None] MFA Rules not processed for user `3c48ccd37a9d4b05952e87e9d3be16a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:41.968920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5f77d685-2f2d-48a0-bf5f-8a2b5624db39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:41.988399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:42.046744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 813/1608] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:42.048229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a4aed-3cc4-4fda-b50c-75f6ec46bdb0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.048548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a4aed-3cc4-4fda-b50c-75f6ec46bdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.048828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a4aed-3cc4-4fda-b50c-75f6ec46bdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.077728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e37a4aed-3cc4-4fda-b50c-75f6ec46bdb0 None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:42.079007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e37a4aed-3cc4-4fda-b50c-75f6ec46bdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:42.080622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 796/1609] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:42.082425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77d685-2f2d-48a0-bf5f-8a2b5624db39 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.099311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77d685-2f2d-48a0-bf5f-8a2b5624db39 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.106526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.107731 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.108207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.108629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] PATH_INFO: `/v3/projects/cb1f77978dd947619db4552c798c7812/users/93bff7193fb34b2997a91164d3853289/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.126884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1f77978dd947619db4552c798c7812, user_id=93bff7193fb34b2997a91164d3853289, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.132292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.156028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 797/1610] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:42 2022] PUT /identity/v3/projects/cb1f77978dd947619db4552c798c7812/users/93bff7193fb34b2997a91164d3853289/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:42.157020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.190532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:42.200280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 814/1611] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Jun 7 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:42.201375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a4aed-3cc4-4fda-b50c-75f6ec46bdb0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.213708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a4aed-3cc4-4fda-b50c-75f6ec46bdb0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.232342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fd25f-d5e5-4e12-954c-2f4ffad3ed9d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.243222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.243943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.244228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.245047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.282610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.283543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.283858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.284150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] PATH_INFO: `/v3/projects/d9f6ff4cee09476c925730a31d32e56f/users/213a7f6737504f9eb89a2e51ddd639ff/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.301874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f6ff4cee09476c925730a31d32e56f, user_id=213a7f6737504f9eb89a2e51ddd639ff, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.304656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.321232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 815/1612] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:42 2022] PUT /identity/v3/projects/d9f6ff4cee09476c925730a31d32e56f/users/213a7f6737504f9eb89a2e51ddd639ff/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:42.322438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.335702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.337355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.349243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 798/1613] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:42.350074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.360747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623972a0-834a-434d-bbe2-9d9778955f2d service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.385318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d339-41b0-4a4c-bf35-f4cf79d7af15 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.388921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eadd2d-1ca1-48bf-afba-9ac13b6d1c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.389567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eadd2d-1ca1-48bf-afba-9ac13b6d1c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.389849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eadd2d-1ca1-48bf-afba-9ac13b6d1c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.390119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eadd2d-1ca1-48bf-afba-9ac13b6d1c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.397095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:42.434747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.436145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.436583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.436901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.437845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.441477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.470763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 799/1614] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:42 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:42.470763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.476551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eadd2d-1ca1-48bf-afba-9ac13b6d1c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.478115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eadd2d-1ca1-48bf-afba-9ac13b6d1c8b service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.490308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 816/1615] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:42.491457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948196-fe68-42d2-a0b0-f66a6a7e0293 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.491748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948196-fe68-42d2-a0b0-f66a6a7e0293 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.492081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948196-fe68-42d2-a0b0-f66a6a7e0293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.519402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ad948196-fe68-42d2-a0b0-f66a6a7e0293 None None] MFA Rules not processed for user `93bff7193fb34b2997a91164d3853289`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:42.520639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ad948196-fe68-42d2-a0b0-f66a6a7e0293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:42.529619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:42.548843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2d365-6454-4bfc-a513-2b029e2449b1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.554558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.555376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.555681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.555976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.557023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.558996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.593565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 800/1616] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:42 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:42.594276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.616545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 817/1617] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:42.619407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa937-5fad-4fa1-ac57-91197323e846 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.620038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa937-5fad-4fa1-ac57-91197323e846 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.621120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa937-5fad-4fa1-ac57-91197323e846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.660076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1f7aa937-5fad-4fa1-ac57-91197323e846 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:42.662037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1f7aa937-5fad-4fa1-ac57-91197323e846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:42.711331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed32a55-9c6e-4afc-9744-439e8c9907e0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.717923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.718873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.719283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.719570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.747918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 818/1618] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:42.749671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa937-5fad-4fa1-ac57-91197323e846 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.762892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa937-5fad-4fa1-ac57-91197323e846 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.771473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.772518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.772802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.773158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.773856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.777683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.805309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.808121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.820839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 819/1619] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:42 2022] POST /identity/v3/users => generated 292 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:42.823412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 801/1620] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:42.825960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.829871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.843874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbba930-2131-452f-8669-1bedd99ec033 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.854172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.857415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.858088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.859640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:42.862951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:42.875596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:42.923002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:42.942311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 820/1621] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:42 2022] POST /identity/v3/users => generated 309 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:42.943600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:42.949338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ac184-9fc1-4e2e-8903-6d063f3d3afc service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:42.957807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:42.958597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:42.958772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:42.959023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.008867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:43.042569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f408d2-4f9c-4b0e-a6a8-47c4d521ce3a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.047788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.048932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.049780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.050311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.062576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.064523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.076908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 821/1622] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:43.078223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.080315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.082694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.094029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6584a4-364a-4ade-8756-26391cf2a52b service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.103076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.104322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.104943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.105071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.106345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.109517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.115081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 802/1623] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:42 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:43.116016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.126857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 822/1624] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:43.129261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd4f44-87b0-48fc-a031-3ccd2d6d7b41 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.138824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.143467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.144492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.145000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.145429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.146314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.149352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.157941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdc803-88e8-4e59-8422-f209701ed332 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.167257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.168465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.168866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.169209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] PATH_INFO: `/v3/projects/b172db494d7d4fff89cc83f56c41eeac/users/e226cf04a03b4743b51d88a9e3b94515/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.180433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 803/1625] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:43.182612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.193027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b172db494d7d4fff89cc83f56c41eeac, user_id=e226cf04a03b4743b51d88a9e3b94515, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.196187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.204331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286651a-1a8a-4a2f-a73e-00d4e5c5d0e5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.216556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 823/1626] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:43 2022] PUT /identity/v3/projects/b172db494d7d4fff89cc83f56c41eeac/users/e226cf04a03b4743b51d88a9e3b94515/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:43.217763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.297125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640899-141a-4004-b582-2c90390dffc5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.303787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.304580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.304925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.305268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.306572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.310491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.334652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 824/1627] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:43.335514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.336660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.337071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.337430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] PATH_INFO: `/v3/projects/66c2317e780b4c35886a1dde96701bc3/users/499f57bf68464669bc43dc83ab8e4f0b/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.337551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.356639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32230da7-c150-4c38-92d1-6f8d83f2cf14 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.363864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42535bb-f42a-450a-80d9-5b8d907c359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.365276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42535bb-f42a-450a-80d9-5b8d907c359f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.365512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42535bb-f42a-450a-80d9-5b8d907c359f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.365790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42535bb-f42a-450a-80d9-5b8d907c359f admin admin] PATH_INFO: `/v3/projects/b172db494d7d4fff89cc83f56c41eeac/users/e226cf04a03b4743b51d88a9e3b94515/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.368499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42535bb-f42a-450a-80d9-5b8d907c359f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b172db494d7d4fff89cc83f56c41eeac, user_id=e226cf04a03b4743b51d88a9e3b94515, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.373096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42535bb-f42a-450a-80d9-5b8d907c359f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.381347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c2317e780b4c35886a1dde96701bc3, user_id=499f57bf68464669bc43dc83ab8e4f0b, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.384273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.394826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 825/1628] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:43 2022] PUT /identity/v3/projects/b172db494d7d4fff89cc83f56c41eeac/users/e226cf04a03b4743b51d88a9e3b94515/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:43.406222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655507c-8d3c-45d6-835f-0c7bd67c4539 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.406543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655507c-8d3c-45d6-835f-0c7bd67c4539 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.406775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655507c-8d3c-45d6-835f-0c7bd67c4539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.428108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 804/1629] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:43 2022] PUT /identity/v3/projects/66c2317e780b4c35886a1dde96701bc3/users/499f57bf68464669bc43dc83ab8e4f0b/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:43.429270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.445979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3655507c-8d3c-45d6-835f-0c7bd67c4539 None None] MFA Rules not processed for user `e226cf04a03b4743b51d88a9e3b94515`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:43.447299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3655507c-8d3c-45d6-835f-0c7bd67c4539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:43.517762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:43.570409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 826/1630] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:43.572580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629aa146-63c5-472a-99d0-f351ea85c182 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.572905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629aa146-63c5-472a-99d0-f351ea85c182 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.573192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629aa146-63c5-472a-99d0-f351ea85c182 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.574775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 827/1631] 10.222.0.18 () {54 vars in 912 bytes} [Tue Jun 7 15:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:58:43.576837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629aa146-63c5-472a-99d0-f351ea85c182 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.579915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd481ff-dcba-4237-a9d7-6bc14a1d4d59 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.590144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acf4cb-4205-43b5-8580-78bda0a0c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.590935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acf4cb-4205-43b5-8580-78bda0a0c130 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.591224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acf4cb-4205-43b5-8580-78bda0a0c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.591502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acf4cb-4205-43b5-8580-78bda0a0c130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.592239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acf4cb-4205-43b5-8580-78bda0a0c130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.594164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acf4cb-4205-43b5-8580-78bda0a0c130 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.623474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 805/1632] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:58:43.628330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f5ab-4545-4c49-a1d5-3f031c17427c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.628825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f5ab-4545-4c49-a1d5-3f031c17427c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.633616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f5ab-4545-4c49-a1d5-3f031c17427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.679661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629aa146-63c5-472a-99d0-f351ea85c182 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.679995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8784f5ab-4545-4c49-a1d5-3f031c17427c None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:43.681180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8784f5ab-4545-4c49-a1d5-3f031c17427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:43.687515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.688990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.691365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.692328 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.710005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.711731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.733939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 828/1633] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:43.736531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.770803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce0536-7182-46c9-bd1c-66fe6e95bb18 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:43.775205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:43.792620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c5e5b-bb61-4da6-a3d5-32b39ef45878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:43.793961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5e5b-bb61-4da6-a3d5-32b39ef45878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.794497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5e5b-bb61-4da6-a3d5-32b39ef45878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.795237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5e5b-bb61-4da6-a3d5-32b39ef45878 admin admin] PATH_INFO: `/v3/projects/66c2317e780b4c35886a1dde96701bc3/users/499f57bf68464669bc43dc83ab8e4f0b/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.798794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c5e5b-bb61-4da6-a3d5-32b39ef45878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c2317e780b4c35886a1dde96701bc3, user_id=499f57bf68464669bc43dc83ab8e4f0b, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:43.803454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c5e5b-bb61-4da6-a3d5-32b39ef45878 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:43.832926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 829/1634] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:43 2022] PUT /identity/v3/projects/66c2317e780b4c35886a1dde96701bc3/users/499f57bf68464669bc43dc83ab8e4f0b/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:43.838866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865b94-b5b9-4efc-a2b5-4094bf692cb5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:43.839160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865b94-b5b9-4efc-a2b5-4094bf692cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:43.839423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865b94-b5b9-4efc-a2b5-4094bf692cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:43.868154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 806/1635] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 15:58:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:43.902648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d4865b94-b5b9-4efc-a2b5-4094bf692cb5 None None] MFA Rules not processed for user `499f57bf68464669bc43dc83ab8e4f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:43.904212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d4865b94-b5b9-4efc-a2b5-4094bf692cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:43.958722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f5ab-4545-4c49-a1d5-3f031c17427c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:43.974710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f5ab-4545-4c49-a1d5-3f031c17427c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:44.032070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 830/1636] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:44.038000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:44.038669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:44.038953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:44.039234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:44.108782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4865b94-b5b9-4efc-a2b5-4094bf692cb5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:44.133397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4865b94-b5b9-4efc-a2b5-4094bf692cb5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:44.136595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:44.138222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:44.143986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f1ec1-7863-45c4-af01-08eddec5b4e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:44.144730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f1ec1-7863-45c4-af01-08eddec5b4e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:44.145016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f1ec1-7863-45c4-af01-08eddec5b4e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:44.149516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f1ec1-7863-45c4-af01-08eddec5b4e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:44.160116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 807/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:58:44.275636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:44.292861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f1ec1-7863-45c4-af01-08eddec5b4e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:44.294674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f1ec1-7863-45c4-af01-08eddec5b4e6 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:44.306992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 831/1638] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:58:44.562194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:44.577562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc27ab-513c-4550-87cf-c72eaeed1776 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:44.586870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678349a-ae58-45d5-9449-61adbd6dcc11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:44.588337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678349a-ae58-45d5-9449-61adbd6dcc11 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:44.588337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678349a-ae58-45d5-9449-61adbd6dcc11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:44.590270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678349a-ae58-45d5-9449-61adbd6dcc11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:44.618990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678349a-ae58-45d5-9449-61adbd6dcc11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:44.621226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678349a-ae58-45d5-9449-61adbd6dcc11 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:44.635639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 808/1639] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:44 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:45.105555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a70955-b7bd-460a-9e9a-daabc1434018 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.106022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a70955-b7bd-460a-9e9a-daabc1434018 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.106103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a70955-b7bd-460a-9e9a-daabc1434018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.215736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-01a70955-b7bd-460a-9e9a-daabc1434018 None None] MFA Rules not processed for user `dfd3065e2d444dc6a4057713e738e0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:45.217454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-01a70955-b7bd-460a-9e9a-daabc1434018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:45.376702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 832/1640] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:45.379482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711eef7-7fdd-4b23-8105-d73b80c9a26c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.379950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711eef7-7fdd-4b23-8105-d73b80c9a26c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.380032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711eef7-7fdd-4b23-8105-d73b80c9a26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.415988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1711eef7-7fdd-4b23-8105-d73b80c9a26c None None] MFA Rules not processed for user `dfd3065e2d444dc6a4057713e738e0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:45.417137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1711eef7-7fdd-4b23-8105-d73b80c9a26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:45.436075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 809/1641] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:45.447479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a70955-b7bd-460a-9e9a-daabc1434018 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:45.466786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a70955-b7bd-460a-9e9a-daabc1434018 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:45.471848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711eef7-7fdd-4b23-8105-d73b80c9a26c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:45.496026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11481f45-9098-41e7-b931-674fabd1db02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:45.498955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11481f45-9098-41e7-b931-674fabd1db02 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.500675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11481f45-9098-41e7-b931-674fabd1db02 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.502389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11481f45-9098-41e7-b931-674fabd1db02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.527969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11481f45-9098-41e7-b931-674fabd1db02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:45.535815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11481f45-9098-41e7-b931-674fabd1db02 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:45.575112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:45.586808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 833/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:58:45.625208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711eef7-7fdd-4b23-8105-d73b80c9a26c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:45.645511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:45.646485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.646976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.647468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] PATH_INFO: `/v3/users/3c48ccd37a9d4b05952e87e9d3be16a4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.648898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c48ccd37a9d4b05952e87e9d3be16a4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:45.650111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:58:45.650111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jun 07 15:58:45.650111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: Jun 07 15:58:45.651692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:45.816052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe847-4af1-48cb-9340-d030ca3a0cbe None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.816273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe847-4af1-48cb-9340-d030ca3a0cbe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.816608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe847-4af1-48cb-9340-d030ca3a0cbe None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.818716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 834/1643] 10.222.0.18 () {54 vars in 961 bytes} [Tue Jun 7 15:58:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:58:45.843774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a16d5-76c0-4b62-b55d-cf30b2058ff0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.843973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a16d5-76c0-4b62-b55d-cf30b2058ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.844509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a16d5-76c0-4b62-b55d-cf30b2058ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.887578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c25a16d5-76c0-4b62-b55d-cf30b2058ff0 None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:45.888939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c25a16d5-76c0-4b62-b55d-cf30b2058ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:45.902895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c48ccd37a9d4b05952e87e9d3be16a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:45.905148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c48ccd37a9d4b05952e87e9d3be16a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:45.932671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 835/1644] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Jun 7 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:58:45.935342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b60186-7f2e-440f-83e4-c8da0ddf7f01 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.935946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b60186-7f2e-440f-83e4-c8da0ddf7f01 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.936798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b60186-7f2e-440f-83e4-c8da0ddf7f01 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.937141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c48ccd37a9d4b05952e87e9d3be16a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:45.943246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c48ccd37a9d4b05952e87e9d3be16a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:45.945608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 836/1645] 10.222.0.18 () {54 vars in 912 bytes} [Tue Jun 7 15:58:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:58:45.949640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:45.952141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:45.952554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:45.970001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e2437a23-84ef-4235-a6a0-da1f9944063f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c48ccd37a9d4b05952e87e9d3be16a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:46.016411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:46.017973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:46.037235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 810/1646] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:58:45 2022] DELETE /identity/v3/users/3c48ccd37a9d4b05952e87e9d3be16a4 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:46.039027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 837/1647] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:46.041940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:46.042227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:46.042506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:46.051526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:46.053071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:46.064295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:46.066935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:46.068325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5700-92e8-42b8-8b1d-342f04be66e2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:46.182245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 811/1648] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:46.185961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db282a12-55f4-4d64-a66d-b860740a000a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:46.186652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282a12-55f4-4d64-a66d-b860740a000a service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:46.186944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282a12-55f4-4d64-a66d-b860740a000a service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:46.187227 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282a12-55f4-4d64-a66d-b860740a000a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:46.227720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:46.232153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:46.407443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:46.418907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d5039-9da5-41b8-a0be-ff319156bf0b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:46.425081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd6b7b-be79-4f99-97bd-1855d1ddc4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:46.427565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd6b7b-be79-4f99-97bd-1855d1ddc4dd service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:46.428588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd6b7b-be79-4f99-97bd-1855d1ddc4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:46.429345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd6b7b-be79-4f99-97bd-1855d1ddc4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:46.434286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db282a12-55f4-4d64-a66d-b860740a000a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:46.435734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db282a12-55f4-4d64-a66d-b860740a000a service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:46.452850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd6b7b-be79-4f99-97bd-1855d1ddc4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:46.454471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd6b7b-be79-4f99-97bd-1855d1ddc4dd service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:46.458422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 838/1649] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:58:46.470247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 812/1650] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:58:46.510387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d9d7b-db71-43fb-9b7a-b838496b4af9 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:46.510754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d9d7b-db71-43fb-9b7a-b838496b4af9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:46.510945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d9d7b-db71-43fb-9b7a-b838496b4af9 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:46.513209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 839/1651] 10.222.0.18 () {54 vars in 912 bytes} [Tue Jun 7 15:58:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:58:46.520701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f120112-5310-4b1a-8648-137399887732 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:46.521004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f120112-5310-4b1a-8648-137399887732 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:46.521407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f120112-5310-4b1a-8648-137399887732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:46.555980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:46.589937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4f120112-5310-4b1a-8648-137399887732 None None] MFA Rules not processed for user `dba9f1fa257a4f44ae159850f9ba349c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:46.591574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4f120112-5310-4b1a-8648-137399887732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:46.604538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 813/1652] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:46.617979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d9d7b-db71-43fb-9b7a-b838496b4af9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:46.634896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d9d7b-db71-43fb-9b7a-b838496b4af9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:46.645889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:46.649235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:46.651450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:46.651963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:46.676711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:46.678674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:46.691570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 840/1653] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:47.219456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f120112-5310-4b1a-8648-137399887732 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:47.222481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4f120112-5310-4b1a-8648-137399887732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:47.279364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f120112-5310-4b1a-8648-137399887732 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:47.301842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:47.304833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:47.305636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:47.305939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] PATH_INFO: `/v3/projects/a59fb3181ede4a16876ce27ed77e6164` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:47.311675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59fb3181ede4a16876ce27ed77e6164)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:47.317874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:47.475291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59fb3181ede4a16876ce27ed77e6164', 'request_id': 'req-1d57b8e2-cfa5-4077-af54-ab32c8243725'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.492023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59fb3181ede4a16876ce27ed77e6164', 'request_id': 'req-1d57b8e2-cfa5-4077-af54-ab32c8243725'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.517093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59fb3181ede4a16876ce27ed77e6164', 'request_id': 'req-1d57b8e2-cfa5-4077-af54-ab32c8243725'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.566473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] The token cache is being invalidate because project a59fb3181ede4a16876ce27ed77e6164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:58:47.571049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.577304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1d57b8e2-cfa5-4077-af54-ab32c8243725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.584657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 814/1654] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:58:47 2022] DELETE /identity/v3/projects/a59fb3181ede4a16876ce27ed77e6164 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:47.589400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:47.609273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3179261-f940-4df9-9c42-14efb3c7ed36 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:47.703669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f159a-53da-4d33-a788-cde9701f7f8e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:47.703966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f159a-53da-4d33-a788-cde9701f7f8e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:47.704973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f159a-53da-4d33-a788-cde9701f7f8e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:47.708641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 815/1655] 10.222.0.18 () {54 vars in 960 bytes} [Tue Jun 7 15:58:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:58:47.717660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb909f27-f031-42ec-b4f8-abbfa320b464 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:47.718154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb909f27-f031-42ec-b4f8-abbfa320b464 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:47.718655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb909f27-f031-42ec-b4f8-abbfa320b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:47.744494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fb909f27-f031-42ec-b4f8-abbfa320b464 None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:47.746291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fb909f27-f031-42ec-b4f8-abbfa320b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:47.749004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:47.750010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:47.750360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:47.750763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] PATH_INFO: `/v3/users/499f57bf68464669bc43dc83ab8e4f0b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:47.752222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499f57bf68464669bc43dc83ab8e4f0b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:47.755349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:47.824253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499f57bf68464669bc43dc83ab8e4f0b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.825379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499f57bf68464669bc43dc83ab8e4f0b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.826209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499f57bf68464669bc43dc83ab8e4f0b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.836048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499f57bf68464669bc43dc83ab8e4f0b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.852831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499f57bf68464669bc43dc83ab8e4f0b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:47.871398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 816/1656] 10.222.0.18 () {58 vars in 1057 bytes} [Tue Jun 7 15:58:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:58:47.872153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 841/1657] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:58:47 2022] DELETE /identity/v3/users/499f57bf68464669bc43dc83ab8e4f0b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:47.872946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb909f27-f031-42ec-b4f8-abbfa320b464 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:47.887667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb909f27-f031-42ec-b4f8-abbfa320b464 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:47.991378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:47.992385 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:47.992867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:47.993352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:47.996087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:48.011262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:48.013802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:48.019972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:48.028606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 817/1658] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:48.159384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:48.162727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:48.290208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:48.292340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:48.304618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:48.313236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417fa65-d66a-4740-b8ca-82eccf8c6c43 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:48.316753 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a015502-9309-4b2b-b15a-cde7201ffa5c service placement] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:48.324863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:48.325941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:48.326455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:48.327298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:48.328462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] PATH_INFO: `/v3/users/28ac0907283a4ba5be34011b2e8e83ec` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:48.328706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:48.329102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:48.330217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] PATH_INFO: `/v3/projects/66c2317e780b4c35886a1dde96701bc3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:48.331555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28ac0907283a4ba5be34011b2e8e83ec)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:48.334854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c2317e780b4c35886a1dde96701bc3)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:48.336625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:48.337591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:48.438270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28ac0907283a4ba5be34011b2e8e83ec'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.440131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28ac0907283a4ba5be34011b2e8e83ec'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.441246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28ac0907283a4ba5be34011b2e8e83ec'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.488499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28ac0907283a4ba5be34011b2e8e83ec'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.500642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c2317e780b4c35886a1dde96701bc3', 'request_id': 'req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.522275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c2317e780b4c35886a1dde96701bc3', 'request_id': 'req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.530707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fa72beca-7d84-4c9d-b580-9ba29bdfa879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28ac0907283a4ba5be34011b2e8e83ec'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.549559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c2317e780b4c35886a1dde96701bc3', 'request_id': 'req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.565685 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] The token cache is being invalidate because project 66c2317e780b4c35886a1dde96701bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:58:48.568676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.569891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2ce7d3e7-7236-4b1c-a88d-cf5bf5b6c7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:48.573927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 818/1659] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:58:48 2022] DELETE /identity/v3/projects/66c2317e780b4c35886a1dde96701bc3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:48.582067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 842/1660] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:58:48 2022] DELETE /identity/v3/users/28ac0907283a4ba5be34011b2e8e83ec => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:48.586923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e5b7d-001e-46c0-b0ba-fc9716baedcb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:48.587218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e5b7d-001e-46c0-b0ba-fc9716baedcb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:48.587505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e5b7d-001e-46c0-b0ba-fc9716baedcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:48.588116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:48.589135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:48.589726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:48.654660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:48.658147 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3a8e5b7d-001e-46c0-b0ba-fc9716baedcb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:48.661798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a8e5b7d-001e-46c0-b0ba-fc9716baedcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:48.665397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:48.820124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 843/1661] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:48.887304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 819/1662] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:48.894659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaf8fd-ae6e-4832-997d-f17db361b8a2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:48.895427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaf8fd-ae6e-4832-997d-f17db361b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:48.896418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaf8fd-ae6e-4832-997d-f17db361b8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:48.925936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:48.927428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:48.929629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4aeaf8fd-ae6e-4832-997d-f17db361b8a2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:48.932053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4aeaf8fd-ae6e-4832-997d-f17db361b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:48.969379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 844/1663] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:48.970398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaf8fd-ae6e-4832-997d-f17db361b8a2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:48.996546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaf8fd-ae6e-4832-997d-f17db361b8a2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:49.006005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:49.006965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.007477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.007989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.009168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:49.011279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:49.013783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:49.048266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 845/1664] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:49.053025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:49.091294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da5ed7-8439-459a-b306-4ec286d59adb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:49.106244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:49.108755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.109631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.110297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.114479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:49.120248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:49.160034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147b0e3-b0d2-4b46-bbc0-f9da0265f492 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:49.181782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:49.182845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.183320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.184100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.212501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 846/1665] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:49 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:49.213654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:49.214878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:49.218546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:49.258535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 820/1666] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:49.359557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:49.417793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c2e2c-9016-4d28-8e78-fc9f862b2b3c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:49.440725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:49.441648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.441890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.442177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.442891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:49.448711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:49.503212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 847/1667] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:49 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:49.507719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:49.525531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f2ff-3e95-4242-ba25-6246692ab3a3 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:49.553781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:49.554708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.563886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.564407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.566240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:49.568188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:49.592552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:49.592780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 821/1668] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:49.631193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0586660-3185-4c9a-ad8f-c6056fbea3ff admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:49.646876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc638228-e868-4f6a-afce-a78e942e6b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:49.649078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc638228-e868-4f6a-afce-a78e942e6b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.651190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc638228-e868-4f6a-afce-a78e942e6b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.651736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc638228-e868-4f6a-afce-a78e942e6b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.655091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc638228-e868-4f6a-afce-a78e942e6b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:49.667057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc638228-e868-4f6a-afce-a78e942e6b86 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:49.685985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 848/1669] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:49.690858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:49.705133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626929b2-1a1b-4861-9e22-8162e63a332d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:49.715649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:49.717054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.717529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.718364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] PATH_INFO: `/v3/projects/6756589e5e78436495317608e07a712e/users/46e0d21ed6224ecfb574c38af39f3431/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.748752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:49.760572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6756589e5e78436495317608e07a712e, user_id=46e0d21ed6224ecfb574c38af39f3431, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:49.786730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:49.870553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 822/1670] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:49 2022] PUT /identity/v3/projects/6756589e5e78436495317608e07a712e/users/46e0d21ed6224ecfb574c38af39f3431/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:49.871460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e043e-526e-4d83-9bf6-6b11d4c51683 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:49.872014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e043e-526e-4d83-9bf6-6b11d4c51683 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:49.880113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e043e-526e-4d83-9bf6-6b11d4c51683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:49.935653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-250e043e-526e-4d83-9bf6-6b11d4c51683 None None] MFA Rules not processed for user `46e0d21ed6224ecfb574c38af39f3431`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:49.937792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-250e043e-526e-4d83-9bf6-6b11d4c51683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:50.049222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 849/1671] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:50.052714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:50.188969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde9fd8-b9c5-40a7-9bea-2868b98bafd7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:50.200894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:50.203075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:50.203587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:50.204061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:50.204969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:50.209402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:50.255543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e043e-526e-4d83-9bf6-6b11d4c51683 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:50.284327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 823/1672] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:50 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:50.288567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e043e-526e-4d83-9bf6-6b11d4c51683 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:50.438026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:50.438733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:50.439318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:50.439417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:50.441662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:50.444420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:50.501235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 850/1673] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:50 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:58:50.503667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:50.520062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7a0c4-6e0a-4c17-b8dc-edef033975f7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:50.536202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:50.543395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:50.544594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:50.544959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:50.546475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:50.548962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:50.585704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 824/1674] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:50.587714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:50.608555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a0cd7-1466-4198-a499-01ce09974e5f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:50.624735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:50.625661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:50.625943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:50.626234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] PATH_INFO: `/v3/projects/225c2f6b572244529da978d48db06337/users/7f9e2898a450441482438e6c10f9bbd1/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:50.666873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=225c2f6b572244529da978d48db06337, user_id=7f9e2898a450441482438e6c10f9bbd1, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:50.672964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:50.710691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 851/1675] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:50 2022] PUT /identity/v3/projects/225c2f6b572244529da978d48db06337/users/7f9e2898a450441482438e6c10f9bbd1/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:50.715117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:50.735213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa0fc6-15d5-496e-ba8e-dd8041f38416 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:50.953551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:50.957415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:50.957769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:50.958050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:50.965776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:50.973151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:51.002176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 825/1676] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:51.006498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:51.030456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfec512-a21d-4616-bc78-58a022cb3c86 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:51.048887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c8668-a900-44fe-83cf-c7bf084b2105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:51.050932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8668-a900-44fe-83cf-c7bf084b2105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:51.053339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8668-a900-44fe-83cf-c7bf084b2105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:51.053923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8668-a900-44fe-83cf-c7bf084b2105 admin admin] PATH_INFO: `/v3/projects/225c2f6b572244529da978d48db06337/users/7f9e2898a450441482438e6c10f9bbd1/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:51.065914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c8668-a900-44fe-83cf-c7bf084b2105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=225c2f6b572244529da978d48db06337, user_id=7f9e2898a450441482438e6c10f9bbd1, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:51.069653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:51.071625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:51.083460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c8668-a900-44fe-83cf-c7bf084b2105 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:51.091504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:51.101904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71d9d1-15bb-4af4-9e5b-db19b9a4784f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:51.136978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 852/1677] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:51 2022] PUT /identity/v3/projects/225c2f6b572244529da978d48db06337/users/7f9e2898a450441482438e6c10f9bbd1/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:51.147294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bffd6-ccf6-46f5-9997-82d81b34cf5e None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:51.147294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bffd6-ccf6-46f5-9997-82d81b34cf5e None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:51.147693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bffd6-ccf6-46f5-9997-82d81b34cf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:51.241291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6c2bffd6-ccf6-46f5-9997-82d81b34cf5e None None] MFA Rules not processed for user `7f9e2898a450441482438e6c10f9bbd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:51.246589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6c2bffd6-ccf6-46f5-9997-82d81b34cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:51.318999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:51.322819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:51.323537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:51.323969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] PATH_INFO: `/v3/projects/fc49db8e3b744ef1972a6b8daff9550b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:51.325388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc49db8e3b744ef1972a6b8daff9550b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:51.329513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:51.331598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:51.458520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc49db8e3b744ef1972a6b8daff9550b', 'request_id': 'req-ed6ebf91-0a55-4f3a-aa7b-49272993256a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:51.474939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc49db8e3b744ef1972a6b8daff9550b', 'request_id': 'req-ed6ebf91-0a55-4f3a-aa7b-49272993256a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:51.502900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc49db8e3b744ef1972a6b8daff9550b', 'request_id': 'req-ed6ebf91-0a55-4f3a-aa7b-49272993256a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:51.528790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 853/1678] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:51.537154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] The token cache is being invalidate because project fc49db8e3b744ef1972a6b8daff9550b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:58:51.538238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:51.539230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:51.548792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2acfa4-712a-4212-83ba-0a8ef5cdd127 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:51.549078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2acfa4-712a-4212-83ba-0a8ef5cdd127 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:51.550578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2acfa4-712a-4212-83ba-0a8ef5cdd127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:51.552712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 826/1679] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:58:51 2022] DELETE /identity/v3/projects/fc49db8e3b744ef1972a6b8daff9550b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:51.555943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:51.565957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:51.609798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8d2acfa4-712a-4212-83ba-0a8ef5cdd127 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:51.611091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8d2acfa4-712a-4212-83ba-0a8ef5cdd127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:51.769469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ebf91-0a55-4f3a-aa7b-49272993256a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:51.780855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 854/1680] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:51.791088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:51.791827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:51.793263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:51.793546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:51.799660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:51.800587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49e7d8-4bd1-4211-9050-bbac409f07dc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:51.802505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49e7d8-4bd1-4211-9050-bbac409f07dc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:51.804146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49e7d8-4bd1-4211-9050-bbac409f07dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:51.865288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2a49e7d8-4bd1-4211-9050-bbac409f07dc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:51.868978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2a49e7d8-4bd1-4211-9050-bbac409f07dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:51.957021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 855/1681] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:51.960996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49e7d8-4bd1-4211-9050-bbac409f07dc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:51.981214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49e7d8-4bd1-4211-9050-bbac409f07dc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:52.003963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:52.004194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:52.005670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:52.005940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:52.006357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:52.006903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:52.009164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:52.012043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:52.017884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:52.033028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 827/1682] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:52.089291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 856/1683] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:52.093134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:52.101081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:52.113685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:52.120486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465dc44-b346-4532-aefc-b9bda577d9d2 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:52.137265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:52.139076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:52.139361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:52.139641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:52.140453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:52.144774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:52.163927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097301d-c78b-43c0-82d6-db6494b5e8f4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:52.198376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:52.199390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:52.201966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:52.202261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] PATH_INFO: `/v3/users/46e0d21ed6224ecfb574c38af39f3431` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:52.209597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e0d21ed6224ecfb574c38af39f3431)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:52.216543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:52.242553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 828/1684] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:52 2022] POST /identity/v3/projects => generated 361 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:52.244500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:52.336149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:52.379546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e0d21ed6224ecfb574c38af39f3431'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:52.380786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e0d21ed6224ecfb574c38af39f3431'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:52.386281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e0d21ed6224ecfb574c38af39f3431'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:52.440964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e0d21ed6224ecfb574c38af39f3431'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:52.442422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:52.477100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5a2b700-aa91-4560-b6c6-f491f33c45d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e0d21ed6224ecfb574c38af39f3431'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:52.513349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 857/1685] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:58:52 2022] DELETE /identity/v3/users/46e0d21ed6224ecfb574c38af39f3431 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:52.518904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f216b7d-532f-467c-9e51-7228c1c48558 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:52.520722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8240104-f4d2-47ac-a4f2-4c3ad9744aaa None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:52.521006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8240104-f4d2-47ac-a4f2-4c3ad9744aaa None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:52.521277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8240104-f4d2-47ac-a4f2-4c3ad9744aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:52.584066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f8240104-f4d2-47ac-a4f2-4c3ad9744aaa None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:52.593405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f8240104-f4d2-47ac-a4f2-4c3ad9744aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:52.720855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:52.722924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:52.723585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:52.724855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:52.726691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:52.730853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:52.742809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 858/1686] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:52.750190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:52.750509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:52.750793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:52.774812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 829/1687] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:52 2022] POST /identity/v3/users => generated 281 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:52.777718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:52.782802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:52.881528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] MFA Rules not processed for user `8348af929ea740f3981631020a219f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:52.882793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:53.056876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e8af6-949c-495d-8dfc-bc46ae45ef2b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:53.075007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25541f3b-b78d-41dd-813c-d7c1f9acefd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:53.075685 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25541f3b-b78d-41dd-813c-d7c1f9acefd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.075964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25541f3b-b78d-41dd-813c-d7c1f9acefd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.076235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25541f3b-b78d-41dd-813c-d7c1f9acefd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:53.122303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25541f3b-b78d-41dd-813c-d7c1f9acefd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:53.123791 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25541f3b-b78d-41dd-813c-d7c1f9acefd1 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:53.164924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 859/1688] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:53.165855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:53.170462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 830/1689] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:53.175643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:53.181927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f843044-46e2-4adb-aeae-1d27187cd9d6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.182188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f843044-46e2-4adb-aeae-1d27187cd9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.182463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f843044-46e2-4adb-aeae-1d27187cd9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:53.243734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169e00-5395-4caa-8ea6-cec4460869bc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:53.257971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:53.258587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.258869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.259145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:53.259881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:53.262236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:53.266131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5f843044-46e2-4adb-aeae-1d27187cd9d6 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:53.267708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5f843044-46e2-4adb-aeae-1d27187cd9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:53.290016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 860/1690] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:53.290016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:53.297817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:53.404453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:53.457987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 831/1691] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 15:58:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:53.461048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f843044-46e2-4adb-aeae-1d27187cd9d6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:53.479203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:53.488693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeefa77-5410-4e49-aeb6-fad72c7d8cd5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:53.490864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f843044-46e2-4adb-aeae-1d27187cd9d6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:53.501787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157d4a8-571f-41e9-87d9-38ee20329993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:53.507175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:53.507865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.509793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.509951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:53.513065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:53.515413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:53.536591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157d4a8-571f-41e9-87d9-38ee20329993 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.537042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157d4a8-571f-41e9-87d9-38ee20329993 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.538549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157d4a8-571f-41e9-87d9-38ee20329993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:53.549739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 832/1692] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:53.550748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:53.557094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157d4a8-571f-41e9-87d9-38ee20329993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:53.573752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34aded-b306-4c62-aeba-81cb52f3eae9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:53.585487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:53.586454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.586735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.588997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] PATH_INFO: `/v3/projects/4c856e2f78294e56b28de9298c5604e9/users/175efbea372d4e5ca369c5bad0822de9/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:53.589582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157d4a8-571f-41e9-87d9-38ee20329993 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:53.630351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c856e2f78294e56b28de9298c5604e9, user_id=175efbea372d4e5ca369c5bad0822de9, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:53.635985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:53.639164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 861/1693] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:53.686158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 833/1694] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:53 2022] PUT /identity/v3/projects/4c856e2f78294e56b28de9298c5604e9/users/175efbea372d4e5ca369c5bad0822de9/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:53.687651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86542c7-c838-4889-9500-2c615b5406a1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.688105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86542c7-c838-4889-9500-2c615b5406a1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.689153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86542c7-c838-4889-9500-2c615b5406a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:53.735813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e86542c7-c838-4889-9500-2c615b5406a1 None None] MFA Rules not processed for user `175efbea372d4e5ca369c5bad0822de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:53.737508 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e86542c7-c838-4889-9500-2c615b5406a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:53.786213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:53.788705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:53.936782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 862/1695] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:53.942084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86542c7-c838-4889-9500-2c615b5406a1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:53.971285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813e498-bea9-47fb-be85-880a0e357d3d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:53.989064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:53.990116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:53.990614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:53.991087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:54.207718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:54.210050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:54.215366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86542c7-c838-4889-9500-2c615b5406a1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:54.225153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 834/1696] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 15:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 15:58:54.243557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:54.244535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:54.245232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:54.252622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:54.256174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:54.265405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:54.363542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 863/1697] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:53 2022] POST /identity/v3/projects => generated 363 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:54.365469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:54.385170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9eeb4-91e7-450e-b8b8-c62347ca132b service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:54.583282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:54.584063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:54.584350 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:54.584656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:54.587574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:54.592086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:54.648787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:54.659545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 835/1698] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:58:54 2022] POST /identity/v3/users => generated 288 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:58:54.663936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:54.672491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:54.679628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:54.680815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed955405-de4e-4271-9896-657ed0a411dc admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:54.700424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:54.701119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:54.701451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:54.701888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:54.702779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:54.706989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:54.725659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5176e5-dc3e-49b1-a07e-91f0f48bb2be admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:54.742278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:54.747053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:54.747777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:54.749116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] PATH_INFO: `/v3/projects/6756589e5e78436495317608e07a712e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:54.751182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 864/1699] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:54.752126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:54.756083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6756589e5e78436495317608e07a712e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:54.766078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:54.775178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b481de2-a83c-4779-bd71-24beb710e75f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:54.792750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:54.793770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:54.794060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:54.794364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] PATH_INFO: `/v3/projects/e052e18929104fcb9322b30bb6ffbff4/users/cb3c5296211a44189700fb9cc731cb08/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:54.833430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e052e18929104fcb9322b30bb6ffbff4, user_id=cb3c5296211a44189700fb9cc731cb08, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:54.837513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:54.882349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 865/1700] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:54 2022] PUT /identity/v3/projects/e052e18929104fcb9322b30bb6ffbff4/users/cb3c5296211a44189700fb9cc731cb08/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:54.886976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:54.954643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:54.961259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6756589e5e78436495317608e07a712e', 'request_id': 'req-e36b8aa6-9376-4d9f-b279-a3ee619ba986'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:54.994400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6756589e5e78436495317608e07a712e', 'request_id': 'req-e36b8aa6-9376-4d9f-b279-a3ee619ba986'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.017475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6756589e5e78436495317608e07a712e', 'request_id': 'req-e36b8aa6-9376-4d9f-b279-a3ee619ba986'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.066849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] The token cache is being invalidate because project 6756589e5e78436495317608e07a712e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:58:55.067197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.070619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.078374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 836/1701] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:58:54 2022] DELETE /identity/v3/projects/6756589e5e78436495317608e07a712e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:55.080117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:55.083860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:55.099225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2741c0-3553-4317-bc40-646fe97e388d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:55.321658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:55.322277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:55.322962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:55.322962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:55.323551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:55.325194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:55.340138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b8aa6-9376-4d9f-b279-a3ee619ba986 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:55.376600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:55.377948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:55.378272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:55.381588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 866/1702] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:58:55.382927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:55.387525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] PATH_INFO: `/v3/users/7f9e2898a450441482438e6c10f9bbd1` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:55.389305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9e2898a450441482438e6c10f9bbd1)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:55.389652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:55.391347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:55.451819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3941f37-2cf5-4c0c-9ac7-973f3e5b739f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:55.474236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a1918-ee86-4dcd-9f48-9acb5d1b8506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:55.481620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1918-ee86-4dcd-9f48-9acb5d1b8506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:55.482324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1918-ee86-4dcd-9f48-9acb5d1b8506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:55.482324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1918-ee86-4dcd-9f48-9acb5d1b8506 admin admin] PATH_INFO: `/v3/projects/e052e18929104fcb9322b30bb6ffbff4/users/cb3c5296211a44189700fb9cc731cb08/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:55.493222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a1918-ee86-4dcd-9f48-9acb5d1b8506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e052e18929104fcb9322b30bb6ffbff4, user_id=cb3c5296211a44189700fb9cc731cb08, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:55.506430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a1918-ee86-4dcd-9f48-9acb5d1b8506 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:55.551251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 867/1703] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:58:55 2022] PUT /identity/v3/projects/e052e18929104fcb9322b30bb6ffbff4/users/cb3c5296211a44189700fb9cc731cb08/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:55.564507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:55.564848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:55.565128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:55.617910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9e2898a450441482438e6c10f9bbd1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.618945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9e2898a450441482438e6c10f9bbd1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.661802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9e2898a450441482438e6c10f9bbd1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.664379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9e2898a450441482438e6c10f9bbd1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.667935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] MFA Rules not processed for user `cb3c5296211a44189700fb9cc731cb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:55.669665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:55.712792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9e2898a450441482438e6c10f9bbd1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:55.824397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:55.862225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 837/1704] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:58:55 2022] DELETE /identity/v3/users/7f9e2898a450441482438e6c10f9bbd1 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:55.974658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 868/1705] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:55.995420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:55.997121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:56.145247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fe609-a132-4667-9499-e86e76d1b93e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:56.157008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545b42-ded8-4bd3-a7ca-0cafc4ec7659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:56.157971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545b42-ded8-4bd3-a7ca-0cafc4ec7659 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:56.158439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545b42-ded8-4bd3-a7ca-0cafc4ec7659 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:56.158924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545b42-ded8-4bd3-a7ca-0cafc4ec7659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:56.398849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db545b42-ded8-4bd3-a7ca-0cafc4ec7659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:56.403444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db545b42-ded8-4bd3-a7ca-0cafc4ec7659 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:56.433700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 838/1706] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:55 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:57.586296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:57.588148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:57.721751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:57.868611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc7ecf-c6b7-4ff4-8882-2d2576577e21 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:57.969328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:57.970300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:57.970726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:58.065911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] MFA Rules not processed for user `dba9f1fa257a4f44ae159850f9ba349c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:58.067143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:58.108788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:58.109665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:58.109954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:58.110291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] PATH_INFO: `/v3/users/175efbea372d4e5ca369c5bad0822de9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:58.122369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] RBAC: Authorizing `identity:delete_user(user_id=175efbea372d4e5ca369c5bad0822de9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:58.127880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:58.261444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175efbea372d4e5ca369c5bad0822de9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.263194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175efbea372d4e5ca369c5bad0822de9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.264420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175efbea372d4e5ca369c5bad0822de9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.284321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175efbea372d4e5ca369c5bad0822de9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.310579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 839/1707] 10.222.0.24 () {58 vars in 1015 bytes} [Tue Jun 7 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:58:58.310889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:58.312548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:58.319737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3a506eba-8789-4199-ba96-aa5fa2ad383d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175efbea372d4e5ca369c5bad0822de9'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.353926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804a0e-d8cc-45ac-a9a3-adf8331c6f5f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:58.360128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 869/1708] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:58:57 2022] DELETE /identity/v3/users/175efbea372d4e5ca369c5bad0822de9 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:58.365348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:58.365348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:58.365348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:58.409816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:58.411063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:58.564089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:58.566133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:58.566458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:58.566978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] PATH_INFO: `/v3/users/93bff7193fb34b2997a91164d3853289` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:58.568403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93bff7193fb34b2997a91164d3853289)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:58.570864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:58.583140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:58.594535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:58.619989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 870/1709] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:58.621059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:58.624059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:58.696378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93bff7193fb34b2997a91164d3853289'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.698701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93bff7193fb34b2997a91164d3853289'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.718616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93bff7193fb34b2997a91164d3853289'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.719872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93bff7193fb34b2997a91164d3853289'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.751315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93bff7193fb34b2997a91164d3853289'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:58.804843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 840/1710] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:58:58 2022] DELETE /identity/v3/users/93bff7193fb34b2997a91164d3853289 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:58.806264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31493bc8-b584-4b35-a1ae-798f06b9b69b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:58.809835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:58.813664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:58.902455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:58.903661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:58.904099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:58.904426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:58.911646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:58.945651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:58.947416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:58.988135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 871/1711] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:58.988948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:58.997316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:59.004094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649eb609-e6f4-413f-8273-428cc4504167 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:59.020746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e266-f213-4ccf-86c7-d111d26468d5 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:59.024388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:59.025775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.026291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.026755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] PATH_INFO: `/v3/projects/225c2f6b572244529da978d48db06337` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.028555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=225c2f6b572244529da978d48db06337)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:59.030577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:59.034821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43260ae0-e768-474d-8c37-7695c8d0ec5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:59.035662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43260ae0-e768-474d-8c37-7695c8d0ec5a service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.035867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43260ae0-e768-474d-8c37-7695c8d0ec5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.036415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43260ae0-e768-474d-8c37-7695c8d0ec5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.043702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-43260ae0-e768-474d-8c37-7695c8d0ec5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:59.075232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43260ae0-e768-474d-8c37-7695c8d0ec5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:59.076861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43260ae0-e768-474d-8c37-7695c8d0ec5a service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:59.110607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 872/1712] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:58:59.113542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.113859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.114165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.170640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:59.171882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:59.173888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225c2f6b572244529da978d48db06337', 'request_id': 'req-f7f09bac-ec96-4316-98c9-cc1f592d8a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:59.195021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225c2f6b572244529da978d48db06337', 'request_id': 'req-f7f09bac-ec96-4316-98c9-cc1f592d8a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:59.206671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225c2f6b572244529da978d48db06337', 'request_id': 'req-f7f09bac-ec96-4316-98c9-cc1f592d8a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:59.239648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] The token cache is being invalidate because project 225c2f6b572244529da978d48db06337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:58:59.239948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:59.240684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f7f09bac-ec96-4316-98c9-cc1f592d8a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:58:59.243128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 841/1713] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:58:58 2022] DELETE /identity/v3/projects/225c2f6b572244529da978d48db06337 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:58:59.247774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adad6d-d1a7-4f13-9cd1-6235e3b62ef7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.248069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adad6d-d1a7-4f13-9cd1-6235e3b62ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.248372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adad6d-d1a7-4f13-9cd1-6235e3b62ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.296481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-29adad6d-d1a7-4f13-9cd1-6235e3b62ef7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:59.297765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-29adad6d-d1a7-4f13-9cd1-6235e3b62ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:59.313030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:59.361813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 873/1714] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:59.511629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:59.513473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:59.544223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 842/1715] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:58:59.550660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce73c0-0340-4806-983c-3ea49d600937 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.550945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce73c0-0340-4806-983c-3ea49d600937 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.551926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce73c0-0340-4806-983c-3ea49d600937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.605444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-85ce73c0-0340-4806-983c-3ea49d600937 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:58:59.610060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-85ce73c0-0340-4806-983c-3ea49d600937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:58:59.652750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:58:59.656981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 843/1716] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:58:59.669326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce73c0-0340-4806-983c-3ea49d600937 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:59.682400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032bf1-c433-498e-a9b6-ab5da3611b23 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:59.700000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce73c0-0340-4806-983c-3ea49d600937 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:59.702222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:59.703011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.703374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.704954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.723757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:59.726879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.727272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.728333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.729504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:58:59.733582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:59.735431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:59.745326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:59.751612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:58:59.785935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 844/1717] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:58:59.790553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:58:59.802985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 874/1718] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:58:59.851900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31df8b-cfb8-42d9-99df-06377d03a607 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:58:59.890265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:58:59.891182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:58:59.891621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:58:59.892057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:58:59.894834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:58:59.912232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:00.027916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 845/1719] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:58:59 2022] POST /identity/v3/projects => generated 399 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:00.033219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:00.051047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27070467-29fa-4673-887f-6516aafe186b service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:00.349586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:00.350232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:00.350520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:00.350799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:00.351502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:00.362556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:00.451003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 875/1720] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:00 2022] POST /identity/v3/users => generated 300 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:00.455164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:00.481403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974f3cb-597a-433d-9d7a-01504ad19ccd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:00.487185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:00.487943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:00.488418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:00.488529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:00.489321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:00.491066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:00.521398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 846/1721] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:00.525470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:00.539731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f2af2-4bf6-4ff2-868c-5afbd9cacf9d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:00.552597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bfa89-2d51-4ede-9cda-3016bcdc6464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:00.559364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bfa89-2d51-4ede-9cda-3016bcdc6464 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:00.559844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bfa89-2d51-4ede-9cda-3016bcdc6464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:00.560300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bfa89-2d51-4ede-9cda-3016bcdc6464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:00.561361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bfa89-2d51-4ede-9cda-3016bcdc6464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:00.563140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bfa89-2d51-4ede-9cda-3016bcdc6464 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:00.602990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 876/1722] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:00.604633 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:00.622519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b064589-68dd-4135-a190-713985b5e29a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:00.632215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:00.633207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:00.633493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:00.633772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] PATH_INFO: `/v3/projects/2ee2256aa05f4bd6b4b54ce5f5f72036/users/76f58a42df4d4449b03e7b37ed25432a/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:00.704584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee2256aa05f4bd6b4b54ce5f5f72036, user_id=76f58a42df4d4449b03e7b37ed25432a, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:00.708733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:00.757150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 847/1723] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:00 2022] PUT /identity/v3/projects/2ee2256aa05f4bd6b4b54ce5f5f72036/users/76f58a42df4d4449b03e7b37ed25432a/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:00.769460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2c48-0ff8-4aa3-9804-3ba1ac55e2e2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:00.771103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2c48-0ff8-4aa3-9804-3ba1ac55e2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:00.771636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2c48-0ff8-4aa3-9804-3ba1ac55e2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:00.881330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e2eb2c48-0ff8-4aa3-9804-3ba1ac55e2e2 None None] MFA Rules not processed for user `76f58a42df4d4449b03e7b37ed25432a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:00.882581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e2eb2c48-0ff8-4aa3-9804-3ba1ac55e2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:01.057101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 877/1724] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:01.070367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:01.233609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e004-df84-486a-a5ee-4fd195e6ab3d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:01.247135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:01.247830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:01.248115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:01.248409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:01.249169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:01.255510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:01.331632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb2c48-0ff8-4aa3-9804-3ba1ac55e2e2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:01.344606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 848/1725] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:01 2022] POST /identity/v3/projects => generated 401 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:01.348824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb2c48-0ff8-4aa3-9804-3ba1ac55e2e2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:01.547900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:01.591747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:01.593986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:01.594352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:01.594650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:01.599851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:01.603655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:01.661362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 878/1726] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:01 2022] POST /identity/v3/users => generated 307 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:01.663375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:01.674950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaaefe-8909-43e6-ac9f-91caa0301a58 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:01.721785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:01.724725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:01.726059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:01.728949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:01.738314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:01.739966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:01.776022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 849/1727] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:01.779605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:01.791500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b432-b34d-4db5-b859-f74f8e64cf07 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:01.822224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:01.827784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:01.828401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:01.828844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] PATH_INFO: `/v3/projects/b2eb6814ed924e45b625ec06124bdc03/users/68a8b83513644f4ab6d0e77f63fd4c77/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:01.880176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2eb6814ed924e45b625ec06124bdc03, user_id=68a8b83513644f4ab6d0e77f63fd4c77, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:01.889399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:01.932849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 879/1728] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:01 2022] PUT /identity/v3/projects/b2eb6814ed924e45b625ec06124bdc03/users/68a8b83513644f4ab6d0e77f63fd4c77/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:01.937063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:01.951545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d437b41-21ef-457d-bb82-fcd2970cca4f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:01.961158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:02.116192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69899f7-1a34-4c14-8dca-d66886221d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:02.117699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69899f7-1a34-4c14-8dca-d66886221d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:02.119300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69899f7-1a34-4c14-8dca-d66886221d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:02.119740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69899f7-1a34-4c14-8dca-d66886221d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:02.122263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69899f7-1a34-4c14-8dca-d66886221d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:02.125579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69899f7-1a34-4c14-8dca-d66886221d8c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:02.166409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 850/1729] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:02.168085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:02.186873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c86d5-2875-4253-8d69-d6582b5baf87 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:02.208891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17758364-42fd-486a-aeae-387cbb8d4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:02.211585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17758364-42fd-486a-aeae-387cbb8d4c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:02.214052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17758364-42fd-486a-aeae-387cbb8d4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:02.214344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17758364-42fd-486a-aeae-387cbb8d4c54 admin admin] PATH_INFO: `/v3/projects/b2eb6814ed924e45b625ec06124bdc03/users/68a8b83513644f4ab6d0e77f63fd4c77/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:02.233884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17758364-42fd-486a-aeae-387cbb8d4c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2eb6814ed924e45b625ec06124bdc03, user_id=68a8b83513644f4ab6d0e77f63fd4c77, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:02.245738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17758364-42fd-486a-aeae-387cbb8d4c54 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:02.289349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 880/1730] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:02 2022] PUT /identity/v3/projects/b2eb6814ed924e45b625ec06124bdc03/users/68a8b83513644f4ab6d0e77f63fd4c77/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:02.295018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:02.295323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:02.295600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:02.322653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2eb00-5880-446d-a036-03e93647f472 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:02.323233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2eb00-5880-446d-a036-03e93647f472 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:02.323658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2eb00-5880-446d-a036-03e93647f472 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:02.325586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 881/1731] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:59:02.349551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:02.350153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:02.350290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:02.393258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] MFA Rules not processed for user `68a8b83513644f4ab6d0e77f63fd4c77`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:02.398733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:02.422155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:02.433546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:02.628608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 851/1732] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:02.631508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:02.632942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:02.776326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95333b80-ebcd-48fc-b881-1ad87c71891b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:02.807893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:02.809283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:02.809889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:02.810332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] PATH_INFO: `/v3/projects/4c856e2f78294e56b28de9298c5604e9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:02.812050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c856e2f78294e56b28de9298c5604e9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:02.814370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:02.844010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 882/1733] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:02.851207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:02.856643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:02.914014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:02.968741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c856e2f78294e56b28de9298c5604e9', 'request_id': 'req-b64bcf8d-74f3-4293-a651-76b26667b77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.002334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c856e2f78294e56b28de9298c5604e9', 'request_id': 'req-b64bcf8d-74f3-4293-a651-76b26667b77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.031412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c856e2f78294e56b28de9298c5604e9', 'request_id': 'req-b64bcf8d-74f3-4293-a651-76b26667b77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.066112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] The token cache is being invalidate because project 4c856e2f78294e56b28de9298c5604e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:03.068730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.071874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.074164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 852/1734] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:02 2022] DELETE /identity/v3/projects/4c856e2f78294e56b28de9298c5604e9 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:03.075140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:03.078329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1739ab-0e40-4677-9401-6505ba6b5285 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:03.079047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:03.257124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:03.259345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:03.259647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:03.259944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:03.267129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:03.326646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bcf8d-74f3-4293-a651-76b26667b77b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:03.339162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:03.344051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:03.348711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:03.349141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:03.350230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:03.360487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:03.417572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:03.420429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:03.468273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 853/1735] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:03.469312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:03.474163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:03.547371 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:03.550071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:03.587226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 883/1736] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:03.719376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b6a81-2e49-4be8-9e04-13e919b8c6a1 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:03.738664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:03.739734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:03.740121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:03.740616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] PATH_INFO: `/v3/users/cb3c5296211a44189700fb9cc731cb08` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:03.744568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3c5296211a44189700fb9cc731cb08)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:03.747030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:03.753015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:03.754646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:03.774853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca48b0-860d-460f-ab29-991c97eb8102 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:03.802301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf938c7-2a80-45cb-9248-b356efe0a547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:03.804664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf938c7-2a80-45cb-9248-b356efe0a547 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:03.807131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf938c7-2a80-45cb-9248-b356efe0a547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:03.832880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf938c7-2a80-45cb-9248-b356efe0a547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:03.837188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ddf938c7-2a80-45cb-9248-b356efe0a547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:03.922640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:03.932210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3c5296211a44189700fb9cc731cb08'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.933044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3c5296211a44189700fb9cc731cb08'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.950333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3c5296211a44189700fb9cc731cb08'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.955434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3c5296211a44189700fb9cc731cb08'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:03.970412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3c5296211a44189700fb9cc731cb08'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:04.004202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf938c7-2a80-45cb-9248-b356efe0a547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:04.007356 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf938c7-2a80-45cb-9248-b356efe0a547 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:04.021961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 854/1737] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:03 2022] DELETE /identity/v3/users/cb3c5296211a44189700fb9cc731cb08 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:04.026548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:04.028776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:04.047193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 884/1738] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:04.272877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fddde-b71d-4005-b71c-9e27d297a47a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:04.287128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:04.290619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:04.292379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:04.292956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] PATH_INFO: `/v3/projects/cb1f77978dd947619db4552c798c7812` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:04.296651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1f77978dd947619db4552c798c7812)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:04.305253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:04.494051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1f77978dd947619db4552c798c7812', 'request_id': 'req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:04.515459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1f77978dd947619db4552c798c7812', 'request_id': 'req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:04.536070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1f77978dd947619db4552c798c7812', 'request_id': 'req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:04.568790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] The token cache is being invalidate because project cb1f77978dd947619db4552c798c7812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:04.569126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:04.570699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-712c66ae-3f7e-4c0e-b334-d5f4a6bad562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:04.574557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 855/1739] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:04 2022] DELETE /identity/v3/projects/cb1f77978dd947619db4552c798c7812 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:04.581445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebfc94-06a5-4ddf-83f2-463a69da04e8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:04.581683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebfc94-06a5-4ddf-83f2-463a69da04e8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:04.582581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebfc94-06a5-4ddf-83f2-463a69da04e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:04.626756 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e7ebfc94-06a5-4ddf-83f2-463a69da04e8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:04.632940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e7ebfc94-06a5-4ddf-83f2-463a69da04e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:04.917987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9bf20-787f-4233-b1bd-a10ff5a6e24b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:04.918262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9bf20-787f-4233-b1bd-a10ff5a6e24b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:04.918558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9bf20-787f-4233-b1bd-a10ff5a6e24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:04.920480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 885/1740] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:04.958371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6bc9bf20-787f-4233-b1bd-a10ff5a6e24b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:04.959627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6bc9bf20-787f-4233-b1bd-a10ff5a6e24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:04.994638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 856/1741] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:04.999103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebfc94-06a5-4ddf-83f2-463a69da04e8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:05.023288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebfc94-06a5-4ddf-83f2-463a69da04e8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:05.041109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:05.041832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:05.061768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:05.062063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:05.062349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:05.076271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:05.076655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:05.077385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:05.077702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:05.079502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:05.140818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 886/1742] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:05.141854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:05.161781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] MFA Rules not processed for user `f47acc995d80476cbd4153e9121e97eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:05.163657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:05.199765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7250036-5703-402e-892b-03d53c6cc1ff admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:05.251827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:05.254480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:05.259671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:05.264642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:05.275986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:05.285870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:05.344950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:05.390309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 887/1743] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:05 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:05.395729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:05.437229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 857/1744] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:05.452305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:05.452466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:05.455382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:05.641034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a37832-68ff-4c4b-9870-1355dc9cce4d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:05.656186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba519d21-f5fb-4ab7-a1d4-1ac6f7a59be1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:05.661882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487beb0a-2c42-4a74-a61c-3c19e7bb87ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:05.662676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487beb0a-2c42-4a74-a61c-3c19e7bb87ab admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:05.665049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487beb0a-2c42-4a74-a61c-3c19e7bb87ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:05.666268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487beb0a-2c42-4a74-a61c-3c19e7bb87ab admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:05.668537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487beb0a-2c42-4a74-a61c-3c19e7bb87ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:05.693396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487beb0a-2c42-4a74-a61c-3c19e7bb87ab admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:05.791241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 888/1745] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:05 2022] POST /identity/v3/users => generated 287 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:05.791241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831dfef-136d-4376-a7ab-77c7a6a2ef26 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:05.791241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831dfef-136d-4376-a7ab-77c7a6a2ef26 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:05.791241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831dfef-136d-4376-a7ab-77c7a6a2ef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:05.821133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:05.821821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:05.822105 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:05.822378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:05.847935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5831dfef-136d-4376-a7ab-77c7a6a2ef26 None None] MFA Rules not processed for user `8348af929ea740f3981631020a219f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:05.849119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5831dfef-136d-4376-a7ab-77c7a6a2ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:06.022697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:06.024474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:06.032719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 889/1746] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:06.033854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831dfef-136d-4376-a7ab-77c7a6a2ef26 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:06.053342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 858/1747] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:06.106829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831dfef-136d-4376-a7ab-77c7a6a2ef26 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:06.135596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:06.138904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:06.141663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:06.146348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:06.147357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:06.157412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:06.209257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 890/1748] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:06.210656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:06.232125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30afebe-a0fd-4bae-bded-ac9863a79ebf service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:06.262778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411f98b-3df8-4929-a180-36befc760788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:06.263465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f98b-3df8-4929-a180-36befc760788 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:06.263751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f98b-3df8-4929-a180-36befc760788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:06.264030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f98b-3df8-4929-a180-36befc760788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:06.264793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411f98b-3df8-4929-a180-36befc760788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:06.266397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411f98b-3df8-4929-a180-36befc760788 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:06.304887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:06.347580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 859/1749] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:06.350441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:06.380198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7d78c-dd15-4bef-ad1d-0cd3008c4d04 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:06.411623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:06.417895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:06.418602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:06.418960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] PATH_INFO: `/v3/projects/70cfc2241ce949a4b18e29d6236d9ace/users/f38eea951904451b88917360ab06c839/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:06.482997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=70cfc2241ce949a4b18e29d6236d9ace, user_id=f38eea951904451b88917360ab06c839, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:06.493965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:06.541979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 891/1750] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:06 2022] PUT /identity/v3/projects/70cfc2241ce949a4b18e29d6236d9ace/users/f38eea951904451b88917360ab06c839/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:06.554921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84a4b0-d841-475f-91ad-7a95a1ed389b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:06.555465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84a4b0-d841-475f-91ad-7a95a1ed389b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:06.556422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84a4b0-d841-475f-91ad-7a95a1ed389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:06.631027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0e84a4b0-d841-475f-91ad-7a95a1ed389b None None] MFA Rules not processed for user `f38eea951904451b88917360ab06c839`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:06.632274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0e84a4b0-d841-475f-91ad-7a95a1ed389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:06.888188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 860/1751] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:07.819033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:07.894233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:07.895526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:07.896015 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:07.975757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] MFA Rules not processed for user `213a7f6737504f9eb89a2e51ddd639ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:07.977767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:08.051539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0d365-230e-4b38-b26b-980b7669a2cf admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:08.081101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:08.083504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:08.083795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:08.089124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:08.129978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:08.147309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:08.148409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:08.200697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 892/1752] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:08.233142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 861/1753] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:08.275973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:08.298428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43e7bb-d77a-4533-94ee-ec32b4699f05 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:08.320739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:08.322032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:08.322285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:08.322842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:08.342880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:08.344989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:08.369863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 893/1754] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:08.541437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:08.543101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:08.771296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c732-531b-42ce-b6e3-d38257ff4488 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:08.786760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:08.792473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:08.792765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:08.793046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] PATH_INFO: `/v3/projects/e052e18929104fcb9322b30bb6ffbff4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:08.795686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e052e18929104fcb9322b30bb6ffbff4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:08.803168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:08.894404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:08.902159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:08.931650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b491b-62f1-49c8-9152-0fff8cc1ba32 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:08.934710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e052e18929104fcb9322b30bb6ffbff4', 'request_id': 'req-adfb3067-072c-48ae-9cdc-8db66b30a33f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:08.952793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e052e18929104fcb9322b30bb6ffbff4', 'request_id': 'req-adfb3067-072c-48ae-9cdc-8db66b30a33f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:08.974196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e052e18929104fcb9322b30bb6ffbff4', 'request_id': 'req-adfb3067-072c-48ae-9cdc-8db66b30a33f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.019504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] The token cache is being invalidate because project e052e18929104fcb9322b30bb6ffbff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:09.020732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.027457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-adfb3067-072c-48ae-9cdc-8db66b30a33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.030406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 862/1755] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:08 2022] DELETE /identity/v3/projects/e052e18929104fcb9322b30bb6ffbff4 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:09.035864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64808-7790-4014-a90a-826de088bb1f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.036410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64808-7790-4014-a90a-826de088bb1f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.036836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64808-7790-4014-a90a-826de088bb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.128037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7be64808-7790-4014-a90a-826de088bb1f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:09.130536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:09.131563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.131765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.132006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] PATH_INFO: `/v3/users/f38eea951904451b88917360ab06c839` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.133315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38eea951904451b88917360ab06c839)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:09.133862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7be64808-7790-4014-a90a-826de088bb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:09.137734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:09.170158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 863/1756] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:09.177878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60357b2e-f9cf-4636-8ec3-ccfaaf28eccd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.178212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60357b2e-f9cf-4636-8ec3-ccfaaf28eccd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.178716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60357b2e-f9cf-4636-8ec3-ccfaaf28eccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.225355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-60357b2e-f9cf-4636-8ec3-ccfaaf28eccd None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:09.233007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-60357b2e-f9cf-4636-8ec3-ccfaaf28eccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:09.247394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38eea951904451b88917360ab06c839'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.248398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38eea951904451b88917360ab06c839'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.249010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38eea951904451b88917360ab06c839'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.268491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 864/1757] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:09.270490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60357b2e-f9cf-4636-8ec3-ccfaaf28eccd None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:09.271225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38eea951904451b88917360ab06c839'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.287212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60357b2e-f9cf-4636-8ec3-ccfaaf28eccd None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:09.301504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:09.302461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.302812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.303213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.304141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:09.305286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:09.307546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-20b43fca-fd64-495c-8483-5ccb09a1748b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38eea951904451b88917360ab06c839'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:09.309031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:09.344584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 894/1758] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:08 2022] DELETE /identity/v3/users/f38eea951904451b88917360ab06c839 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:09.349893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.350091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 865/1759] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:09.350584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.351026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.354300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:09.356277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:09.376644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccf061-52de-4bb6-9973-fabda0c96d7b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:09.382751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:09.386180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:09.538062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 895/1760] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:09.542996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:09.543938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.543938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.544214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.544955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:09.546877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:09.608622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 866/1761] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:09 2022] POST /identity/v3/projects => generated 365 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:09.613080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:09.613266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:09.614636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:09.623954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:09.735711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b460a2-cf4e-4790-83bb-4693fd0e96d5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:09.805449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216d8df-9dbc-4a3a-ad38-54cb339425e5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:09.833149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:09.833834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.834125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.834404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.835106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:09.837242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:09.842926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:09.844054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.844551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.844994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.869687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:09.871473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:09.884629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:09.899051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 896/1762] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:09.899237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 867/1763] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:09 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:09.905168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:09.936635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef868ec-5f1f-442d-be7a-b287b8b81865 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:09.954565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:09.955766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:09.956044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:09.956295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:09.965464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:09.969150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:09.995947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:10.008543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb74b5-f081-4563-981f-c1825c9eb900 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:10.010199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 897/1764] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:10.025168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419ff14-1313-43bd-b750-f6b01eb5059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:10.025643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419ff14-1313-43bd-b750-f6b01eb5059b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:10.025919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419ff14-1313-43bd-b750-f6b01eb5059b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:10.026356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419ff14-1313-43bd-b750-f6b01eb5059b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:10.027240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419ff14-1313-43bd-b750-f6b01eb5059b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:10.029053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419ff14-1313-43bd-b750-f6b01eb5059b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:10.059003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:10.070446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 868/1765] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:10.077357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe7bb2-22e3-48c4-a6dc-319182667ae0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:10.088875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:10.092793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:10.093860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:10.094256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] PATH_INFO: `/v3/projects/9cf0d0db896b489784c6dcb15982a342/users/ef2802d2eb884402b87fff1a8149ef98/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:10.135780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf0d0db896b489784c6dcb15982a342, user_id=ef2802d2eb884402b87fff1a8149ef98, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:10.144465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:10.183308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 898/1766] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:10 2022] PUT /identity/v3/projects/9cf0d0db896b489784c6dcb15982a342/users/ef2802d2eb884402b87fff1a8149ef98/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:10.196257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfe1ba-1aa6-4794-9492-3e2332bbff94 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:10.197374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfe1ba-1aa6-4794-9492-3e2332bbff94 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:10.197671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfe1ba-1aa6-4794-9492-3e2332bbff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:10.254400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-18cfe1ba-1aa6-4794-9492-3e2332bbff94 None None] MFA Rules not processed for user `ef2802d2eb884402b87fff1a8149ef98`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:10.255972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-18cfe1ba-1aa6-4794-9492-3e2332bbff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:10.395641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:10.397779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:10.412541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 869/1767] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:10.414133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfe1ba-1aa6-4794-9492-3e2332bbff94 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:10.561199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfe1ba-1aa6-4794-9492-3e2332bbff94 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:10.574541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd7772-8922-4778-b52e-11e8005aae07 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:10.591742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:10.592679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:10.592969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:10.593101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:10.593748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] PATH_INFO: `/v3/users/76f58a42df4d4449b03e7b37ed25432a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:10.593859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:10.594050 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:10.594326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:10.595029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:10.595242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f58a42df4d4449b03e7b37ed25432a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:10.598044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:10.600333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:10.649345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:10.668308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 870/1768] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:10 2022] POST /identity/v3/projects => generated 365 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:10.669112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:10.701606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f58a42df4d4449b03e7b37ed25432a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:10.706500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f58a42df4d4449b03e7b37ed25432a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:10.712832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f58a42df4d4449b03e7b37ed25432a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:10.727991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f58a42df4d4449b03e7b37ed25432a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:10.761822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f5f17b10-ab54-41ef-a0c7-7e35ae12b7a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f58a42df4d4449b03e7b37ed25432a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:10.790534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 899/1769] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:10 2022] DELETE /identity/v3/users/76f58a42df4d4449b03e7b37ed25432a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:10.793657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:10.793943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:10.794219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:10.834124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:10.835394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:10.926234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9049-5787-4ef6-8f8e-1493c8f285ba admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:10.937846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:11.001993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 900/1770] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:11.033827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:11.035430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:11.075110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:11.075445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.075844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.076215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.077022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:11.081970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:11.148715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 871/1771] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:10 2022] POST /identity/v3/users => generated 289 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:11.149682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:11.153568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:11.192622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd662ae-8b12-4e2d-bd76-fc322a1520b7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:11.202312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:11.202931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.203323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.203650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.204481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5342c-25fa-4f4a-89c3-70e90d8a4e89 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:11.204612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:11.222333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:11.222725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.223010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.223281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.225308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:11.257873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:11.260115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 872/1772] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:11.263496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:11.264022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:11.292738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 901/1773] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:11.309923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99cd2f-d9e1-40c8-b8d5-b0a61f662251 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:11.328191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:11.330820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.331509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.333730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] PATH_INFO: `/v3/projects/3fc6083db1ff4d458c882d17f400087b/users/3483b26a20bb4e05badcc78195364b84/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.380944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc6083db1ff4d458c882d17f400087b, user_id=3483b26a20bb4e05badcc78195364b84, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:11.383573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:11.423209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:11.429153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 873/1774] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:11 2022] PUT /identity/v3/projects/3fc6083db1ff4d458c882d17f400087b/users/3483b26a20bb4e05badcc78195364b84/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:11.438319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c726cd-9a47-4fbd-b8a7-64ae6c4ea3ba service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:11.542268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:11.568825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562eb7f-98fd-4daa-a4da-fd2a14db3012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:11.571370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562eb7f-98fd-4daa-a4da-fd2a14db3012 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.571370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562eb7f-98fd-4daa-a4da-fd2a14db3012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.571370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562eb7f-98fd-4daa-a4da-fd2a14db3012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.571370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562eb7f-98fd-4daa-a4da-fd2a14db3012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:11.572939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562eb7f-98fd-4daa-a4da-fd2a14db3012 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:11.597952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:11.598839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 902/1775] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:11.614083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1748f-05dd-43cb-b797-b6be0a4157b8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:11.628792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:11.629825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.630122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.630402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] PATH_INFO: `/v3/projects/3fc6083db1ff4d458c882d17f400087b/users/3483b26a20bb4e05badcc78195364b84/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.653511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc6083db1ff4d458c882d17f400087b, user_id=3483b26a20bb4e05badcc78195364b84, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:11.656591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:11.674854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 874/1776] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:11 2022] PUT /identity/v3/projects/3fc6083db1ff4d458c882d17f400087b/users/3483b26a20bb4e05badcc78195364b84/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:11.686629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba253e-7262-46ab-86b0-dd4bed301199 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.686906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba253e-7262-46ab-86b0-dd4bed301199 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.687204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba253e-7262-46ab-86b0-dd4bed301199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.689361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:11.691475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:11.766173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-26ba253e-7262-46ab-86b0-dd4bed301199 None None] MFA Rules not processed for user `3483b26a20bb4e05badcc78195364b84`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:11.767567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-26ba253e-7262-46ab-86b0-dd4bed301199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:11.798418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e1f1c-3820-4f22-a709-3da41b9e2401 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:11.806276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:11.807935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.808226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.808587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] PATH_INFO: `/v3/projects/70cfc2241ce949a4b18e29d6236d9ace` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.810885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70cfc2241ce949a4b18e29d6236d9ace)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:11.817771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:11.923506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70cfc2241ce949a4b18e29d6236d9ace', 'request_id': 'req-039b9196-babd-4955-90ee-f47e2f5e281c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:11.927931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:11.934889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70cfc2241ce949a4b18e29d6236d9ace', 'request_id': 'req-039b9196-babd-4955-90ee-f47e2f5e281c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:11.938697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 903/1777] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:11.941194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befc105-a9c3-4635-b1a8-99d580ec24d0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:11.941601 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befc105-a9c3-4635-b1a8-99d580ec24d0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:11.941998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befc105-a9c3-4635-b1a8-99d580ec24d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:11.958575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70cfc2241ce949a4b18e29d6236d9ace', 'request_id': 'req-039b9196-babd-4955-90ee-f47e2f5e281c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:11.964641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0befc105-a9c3-4635-b1a8-99d580ec24d0 None None] MFA Rules not processed for user `f47acc995d80476cbd4153e9121e97eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:11.966093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0befc105-a9c3-4635-b1a8-99d580ec24d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:11.974715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] The token cache is being invalidate because project 70cfc2241ce949a4b18e29d6236d9ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:11.975113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:11.975840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:11.978961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 875/1778] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:11 2022] DELETE /identity/v3/projects/70cfc2241ce949a4b18e29d6236d9ace => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:11.980227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:11.983273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:12.049388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 904/1779] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:12.050972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95ee3e-ec62-491a-b19f-3a2a9a8451cb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.051597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95ee3e-ec62-491a-b19f-3a2a9a8451cb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.051950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95ee3e-ec62-491a-b19f-3a2a9a8451cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.075425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6d95ee3e-ec62-491a-b19f-3a2a9a8451cb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:12.077810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6d95ee3e-ec62-491a-b19f-3a2a9a8451cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:12.082356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9196-babd-4955-90ee-f47e2f5e281c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.087377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.088057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.088336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.088653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.090784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:12.192681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 905/1780] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:12.199102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efc0f-3d07-49e8-9066-8f1bf94438bc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.200684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efc0f-3d07-49e8-9066-8f1bf94438bc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.201188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efc0f-3d07-49e8-9066-8f1bf94438bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.210815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.213396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.225053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-cc3efc0f-3d07-49e8-9066-8f1bf94438bc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:12.226273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cc3efc0f-3d07-49e8-9066-8f1bf94438bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:12.237291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 906/1781] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:12.238062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 876/1782] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:12.257100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3efc0f-3d07-49e8-9066-8f1bf94438bc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.279573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3efc0f-3d07-49e8-9066-8f1bf94438bc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.301171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.303336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.303640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.304489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.305450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:12.305760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.307534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.319094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:12.362120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.363008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 907/1783] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:12.374092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.376310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4928-36c8-4141-af74-8575e320ce89 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.390984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.391660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.392401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.392682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.398150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:12.400490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fd107-f274-4c0f-b2af-8e4419bfc193 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.410667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.412182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.412777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.413253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.414452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.417574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.473651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 908/1784] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:12 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:12.474510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.550349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.553971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.587009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 877/1785] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:12.627811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580adde-1142-4227-8ac1-cf022b5b3830 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.640702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.643537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.643845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.644653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.647127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.651689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.697736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 909/1786] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:12 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:12.700428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.710560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.714535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:12.721066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da568f-3708-45e9-83de-23a9456a02a5 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.729543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.730227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.730498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.730836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.731484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.733112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.743841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb831b-9ee9-4e2a-b750-ec029446afa7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.754804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 878/1787] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:12.757364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.758418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.758704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.758975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] PATH_INFO: `/v3/projects/2ee2256aa05f4bd6b4b54ce5f5f72036` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.760595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee2256aa05f4bd6b4b54ce5f5f72036)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.761609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.762337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: WARNING py.warnings [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 15:59:12.762337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: warnings.warn(msg) Jun 07 15:59:12.762337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: Jun 07 15:59:12.763520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.782690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6b909-0c58-4497-a7bf-45c56d05ade8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.792091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:12.793099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:12.793486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:12.794458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:12.795681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:12.797658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:12.821152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 879/1788] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:12.824001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.878044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17d32a-da72-4f74-8f4f-476c3ba9075f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:12.890166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:12.901825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee2256aa05f4bd6b4b54ce5f5f72036', 'request_id': 'req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:12.911410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee2256aa05f4bd6b4b54ce5f5f72036', 'request_id': 'req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:12.936973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee2256aa05f4bd6b4b54ce5f5f72036', 'request_id': 'req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:12.957741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] The token cache is being invalidate because project 2ee2256aa05f4bd6b4b54ce5f5f72036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:12.958114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:12.959584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:12.964668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 910/1789] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:12 2022] DELETE /identity/v3/projects/2ee2256aa05f4bd6b4b54ce5f5f72036 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:12.966977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:12.968868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:12.999571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb0c3e-14bb-48f5-8b7e-2668e24ff9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:13.002320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0c3e-14bb-48f5-8b7e-2668e24ff9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:13.002948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0c3e-14bb-48f5-8b7e-2668e24ff9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:13.003567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0c3e-14bb-48f5-8b7e-2668e24ff9f6 admin admin] PATH_INFO: `/v3/projects/2cea31c70e1a40c790e4d1dcd6b7410d/users/adb6ac0134cf4740b03db33cfaeb4457/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:13.030860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb0c3e-14bb-48f5-8b7e-2668e24ff9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cea31c70e1a40c790e4d1dcd6b7410d, user_id=adb6ac0134cf4740b03db33cfaeb4457, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:13.038953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:13.056309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb0c3e-14bb-48f5-8b7e-2668e24ff9f6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:13.059020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc1a75-c0ea-4279-912d-9a9c31f6c962 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:13.066068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:13.067573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:13.068215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:13.069720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] PATH_INFO: `/v3/users/68a8b83513644f4ab6d0e77f63fd4c77` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:13.071956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a8b83513644f4ab6d0e77f63fd4c77)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:13.074012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:13.097996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 880/1790] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:12 2022] PUT /identity/v3/projects/2cea31c70e1a40c790e4d1dcd6b7410d/users/adb6ac0134cf4740b03db33cfaeb4457/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:13.100052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:13.100577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:13.101073 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:13.152989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:13.154646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:13.185363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a8b83513644f4ab6d0e77f63fd4c77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:13.185603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a8b83513644f4ab6d0e77f63fd4c77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:13.186049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a8b83513644f4ab6d0e77f63fd4c77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:13.195999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a8b83513644f4ab6d0e77f63fd4c77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:13.226097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8cb676fb-31d1-4afc-8b17-4d63e6315f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a8b83513644f4ab6d0e77f63fd4c77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:13.254139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 911/1791] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:12 2022] DELETE /identity/v3/users/68a8b83513644f4ab6d0e77f63fd4c77 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:13.254139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:13.254139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:13.254469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:13.255799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 881/1792] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:13.277504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:13.279818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] MFA Rules not processed for user `213a7f6737504f9eb89a2e51ddd639ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:13.280771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:13.281682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:13.383867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 912/1793] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:13.490383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2038c-218f-41a6-a81e-0a59b66402e9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:13.564763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:13.571164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca7197-2899-462d-92b2-a2453f57b316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:13.571850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7197-2899-462d-92b2-a2453f57b316 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:13.572133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7197-2899-462d-92b2-a2453f57b316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:13.572813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7197-2899-462d-92b2-a2453f57b316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:13.665180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca7197-2899-462d-92b2-a2453f57b316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:13.666907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca7197-2899-462d-92b2-a2453f57b316 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:13.689766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 882/1794] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:13.779642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:13.782066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:13.827456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c1270-135d-4ca9-84c8-82f934af9b8a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:13.839167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:13.840100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:13.840313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:13.840611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] PATH_INFO: `/v3/projects/b2eb6814ed924e45b625ec06124bdc03` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:13.842035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2eb6814ed924e45b625ec06124bdc03)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:13.843932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:14.012390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2eb6814ed924e45b625ec06124bdc03', 'request_id': 'req-8d5edbd7-d81f-4b17-b405-620e795fde93'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:14.025438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2eb6814ed924e45b625ec06124bdc03', 'request_id': 'req-8d5edbd7-d81f-4b17-b405-620e795fde93'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:14.043314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2eb6814ed924e45b625ec06124bdc03', 'request_id': 'req-8d5edbd7-d81f-4b17-b405-620e795fde93'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:14.046515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:14.110146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] The token cache is being invalidate because project b2eb6814ed924e45b625ec06124bdc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:14.113247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:14.114655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d5edbd7-d81f-4b17-b405-620e795fde93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:14.122822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 913/1795] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:13 2022] DELETE /identity/v3/projects/b2eb6814ed924e45b625ec06124bdc03 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:14.124514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1ff07-dbbb-4478-9a69-e94abbf94b1d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.125425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1ff07-dbbb-4478-9a69-e94abbf94b1d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.125837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1ff07-dbbb-4478-9a69-e94abbf94b1d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.128248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3f47-615c-482e-9142-8372b63df121 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.128447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 883/1796] 10.222.0.18 () {54 vars in 912 bytes} [Tue Jun 7 15:59:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:59:14.128662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3f47-615c-482e-9142-8372b63df121 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.129037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3f47-615c-482e-9142-8372b63df121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.140077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.140395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.142955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.153656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8a4a3f47-615c-482e-9142-8372b63df121 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:14.155194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8a4a3f47-615c-482e-9142-8372b63df121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:14.218957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] MFA Rules not processed for user `dba9f1fa257a4f44ae159850f9ba349c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:14.220196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:14.328713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 914/1797] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:14.331270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1794d8b-c2a3-4f87-86b3-a2accb82caa8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.332720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1794d8b-c2a3-4f87-86b3-a2accb82caa8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.333518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1794d8b-c2a3-4f87-86b3-a2accb82caa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.359468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b1794d8b-c2a3-4f87-86b3-a2accb82caa8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:14.362340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b1794d8b-c2a3-4f87-86b3-a2accb82caa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:14.376490 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 915/1798] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:14.378576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1794d8b-c2a3-4f87-86b3-a2accb82caa8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:14.402779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1794d8b-c2a3-4f87-86b3-a2accb82caa8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:14.425892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:14.426623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.426883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.452879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 884/1799] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:14.456615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.457238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:14.458256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:14.459283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:14.459395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:14.464580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:14.477310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6e294-9352-447a-90e7-6d043df4ebfd None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:14.488736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:14.489698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.490010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.490291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.493460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 916/1800] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:14.497900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:14.507468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:14.510790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:14.520386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd1d9b-c443-429c-87bd-aadb98ebb02a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:14.531517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 885/1801] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:14.532518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:14.533192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.533512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.533793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.534719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:14.536832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:14.576947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 917/1802] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:14 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:14.581365 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:14.598297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe602-3749-4933-b759-ac492d39431b service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:14.683569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:14.684537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.684896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.685267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.687158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:14.694510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:14.747457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 886/1803] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:14 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:14.749746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:14.765311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7235d95-73fb-44aa-8f07-a4b0a041ac8b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:14.782285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:14.783225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.787687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.788087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.790600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:14.792901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:14.832968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 918/1804] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:14.837480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:14.874500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9abef-52b8-44f4-9847-fbca4c36403d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:14.904606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eebe24-8b72-42ed-a616-2032b2ee356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:14.906537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebe24-8b72-42ed-a616-2032b2ee356f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:14.906832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebe24-8b72-42ed-a616-2032b2ee356f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:14.907117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebe24-8b72-42ed-a616-2032b2ee356f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:14.907930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eebe24-8b72-42ed-a616-2032b2ee356f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:14.910081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eebe24-8b72-42ed-a616-2032b2ee356f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:14.976647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 887/1805] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:14.981524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.015527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9259-4e4e-4c82-b488-51ae08283b9f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.029873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239042a9-b5ef-4807-aab7-08b83fad9eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.031735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239042a9-b5ef-4807-aab7-08b83fad9eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.032005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239042a9-b5ef-4807-aab7-08b83fad9eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.032279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239042a9-b5ef-4807-aab7-08b83fad9eb3 admin admin] PATH_INFO: `/v3/projects/22d284b9f8c841828a73483f2efec06d/users/e258f145914243d4a17a0cd474de8e77/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.093199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239042a9-b5ef-4807-aab7-08b83fad9eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d284b9f8c841828a73483f2efec06d, user_id=e258f145914243d4a17a0cd474de8e77, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.097204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239042a9-b5ef-4807-aab7-08b83fad9eb3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.115053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 919/1806] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:14 2022] PUT /identity/v3/projects/22d284b9f8c841828a73483f2efec06d/users/e258f145914243d4a17a0cd474de8e77/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:15.125715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13388-f39a-487f-8128-aa43d0930522 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.126210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13388-f39a-487f-8128-aa43d0930522 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.126282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13388-f39a-487f-8128-aa43d0930522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.172571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:15.192190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-92a13388-f39a-487f-8128-aa43d0930522 None None] MFA Rules not processed for user `e258f145914243d4a17a0cd474de8e77`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:15.193571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-92a13388-f39a-487f-8128-aa43d0930522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:15.270284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.270932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.271294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.325140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] MFA Rules not processed for user `adb6ac0134cf4740b03db33cfaeb4457`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:15.326396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:15.351007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 888/1807] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:15.362322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a13388-f39a-487f-8128-aa43d0930522 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.463156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a13388-f39a-487f-8128-aa43d0930522 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.470763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.471336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.471624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.471904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.472649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.474593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.485699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 920/1808] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:15.490886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.498873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:15.524345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf5c30-e043-470a-8ef0-10d9d692ee70 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.537104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 889/1809] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:15 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:15.537746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.538711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.538890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.539186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.539592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.543733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.553153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.559858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:15.602029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 921/1810] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:15 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:15.605131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.700893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91cece-7b9f-42a8-b187-8d99557e39fa admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.712125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6009a8-0305-4de2-b320-e498376705fd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.715383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.719329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.720196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.720396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.720747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.721041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.721745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.722789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.723221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.723887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.724241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.726314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.746525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 890/1811] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:15 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:15.750224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.755854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 922/1812] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:15 2022] POST /identity/v3/users => generated 308 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:15.759395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.763403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e913eb-f976-4fac-9b9d-87af317d80c9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.770300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.770939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.771352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.771638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.772454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.772942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f321cf-fa67-46dd-8fda-d34d0ac01b54 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.774305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.779111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.779884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.780201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.780647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.781277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.783966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.791257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 891/1813] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:15.792542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.797897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 923/1814] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:15.802512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.803392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:15.807831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f841640-a3dd-48a0-9a34-9606f8446b8d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.813877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.815380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.815676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.816166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] PATH_INFO: `/v3/projects/2e1c9c1858334efe85bdec9da1ade35f/users/f90a5e161a3a4978829ee4e261ae60de/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.817670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9731416-34fb-4508-8da1-006eb73a82e8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.824119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.825548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:15.825997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:15.826410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] PATH_INFO: `/v3/projects/3641a2c142c74c25aa8212c7ee592d5f/users/cefadc82cf2c4bac8ed508f9d26b2a27/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:15.836400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1c9c1858334efe85bdec9da1ade35f, user_id=f90a5e161a3a4978829ee4e261ae60de, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.839889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.845992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3641a2c142c74c25aa8212c7ee592d5f, user_id=cefadc82cf2c4bac8ed508f9d26b2a27, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:15.848841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:15.869877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 924/1815] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:15 2022] PUT /identity/v3/projects/3641a2c142c74c25aa8212c7ee592d5f/users/cefadc82cf2c4bac8ed508f9d26b2a27/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:15.874544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.884118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 892/1816] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:15 2022] PUT /identity/v3/projects/2e1c9c1858334efe85bdec9da1ade35f/users/f90a5e161a3a4978829ee4e261ae60de/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:15.885516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:15.994053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0d921-94d7-4e20-bdaa-00862ec76871 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.995133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68788b8d-0ca5-4968-9ba2-e2b664af140c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:15.999011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:15.999810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.000093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.000387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.001116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:16.002214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:16.002975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.003286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.003570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.004307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:16.006002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:16.013860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:16.016960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 925/1817] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:16.022559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:16.032239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 893/1818] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:16.032724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592f7e6-6d1c-4bf5-a62c-a227761acf62 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:16.034303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:16.038252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014ebc6-0998-48e7-959e-35fceb9919b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:16.039130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ebc6-0998-48e7-959e-35fceb9919b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.039409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ebc6-0998-48e7-959e-35fceb9919b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.039682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ebc6-0998-48e7-959e-35fceb9919b0 admin admin] PATH_INFO: `/v3/projects/3641a2c142c74c25aa8212c7ee592d5f/users/cefadc82cf2c4bac8ed508f9d26b2a27/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.041739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014ebc6-0998-48e7-959e-35fceb9919b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3641a2c142c74c25aa8212c7ee592d5f, user_id=cefadc82cf2c4bac8ed508f9d26b2a27, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:16.043694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e22b3-73c5-48d1-affc-90638ce1391e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:16.044408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014ebc6-0998-48e7-959e-35fceb9919b0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:16.048331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:16.049523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.049811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.050092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] PATH_INFO: `/v3/projects/2e1c9c1858334efe85bdec9da1ade35f/users/f90a5e161a3a4978829ee4e261ae60de/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.052145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1c9c1858334efe85bdec9da1ade35f, user_id=f90a5e161a3a4978829ee4e261ae60de, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:16.054804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:16.055663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 926/1819] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:16 2022] PUT /identity/v3/projects/3641a2c142c74c25aa8212c7ee592d5f/users/cefadc82cf2c4bac8ed508f9d26b2a27/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:16.066024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 894/1820] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:16 2022] PUT /identity/v3/projects/2e1c9c1858334efe85bdec9da1ade35f/users/f90a5e161a3a4978829ee4e261ae60de/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:16.076428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.076722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.077004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.114939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] MFA Rules not processed for user `f90a5e161a3a4978829ee4e261ae60de`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:16.116153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:16.132752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:16.230173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 927/1821] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:16.238391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:16.241037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:16.320148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfae52-1154-46da-a5e2-bd3b952e18c4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:16.324341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c3168-1c59-4a91-9ab2-b478a8984503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:16.325154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c3168-1c59-4a91-9ab2-b478a8984503 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.325684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c3168-1c59-4a91-9ab2-b478a8984503 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.326330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c3168-1c59-4a91-9ab2-b478a8984503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.337536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c3168-1c59-4a91-9ab2-b478a8984503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:16.339264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c3168-1c59-4a91-9ab2-b478a8984503 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:16.351652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 895/1822] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:16.383965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:16.385473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:16.474499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172ec34-9b21-4a57-a6ad-010f1745d112 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:16.479898 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:16.480966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.481345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.481679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] PATH_INFO: `/v3/users/ef2802d2eb884402b87fff1a8149ef98` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.482959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2802d2eb884402b87fff1a8149ef98)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:16.485063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:16.558953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2802d2eb884402b87fff1a8149ef98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:16.559816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2802d2eb884402b87fff1a8149ef98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:16.560753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2802d2eb884402b87fff1a8149ef98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:16.571646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2802d2eb884402b87fff1a8149ef98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:16.606084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2802d2eb884402b87fff1a8149ef98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:16.628774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 928/1823] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:16 2022] DELETE /identity/v3/users/ef2802d2eb884402b87fff1a8149ef98 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:16.632397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.633295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.633295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.651863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:16.653351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:16.733908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 896/1824] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:16.739934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:16.741429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:16.798981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:16.840152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570c210-921a-414d-97f9-de9b2b7c28f1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:16.844713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ddb6d-5351-457b-915f-e0f0b44edbe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:16.844890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ddb6d-5351-457b-915f-e0f0b44edbe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:16.845129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ddb6d-5351-457b-915f-e0f0b44edbe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:16.845701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ddb6d-5351-457b-915f-e0f0b44edbe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:16.859786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ddb6d-5351-457b-915f-e0f0b44edbe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:16.861425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ddb6d-5351-457b-915f-e0f0b44edbe2 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:16.874905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 929/1825] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:17.144254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:17.145811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:17.159931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e2bd8-de25-4b6e-b7bb-506904321f26 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:17.164656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:17.165541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:17.165820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:17.166198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] PATH_INFO: `/v3/users/e258f145914243d4a17a0cd474de8e77` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:17.167290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e258f145914243d4a17a0cd474de8e77)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:17.170320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:17.172010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:17.224597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e258f145914243d4a17a0cd474de8e77'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.225511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e258f145914243d4a17a0cd474de8e77'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.234983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e258f145914243d4a17a0cd474de8e77'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.235819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e258f145914243d4a17a0cd474de8e77'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.251135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-153a84f9-f49e-4c9f-ac4f-13f89aca09c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e258f145914243d4a17a0cd474de8e77'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.272972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 897/1826] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:17 2022] DELETE /identity/v3/users/e258f145914243d4a17a0cd474de8e77 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:17.278343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:17.278821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:17.278924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:17.301504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:17.303049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:17.306751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:17.307156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:17.307411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:17.342953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] MFA Rules not processed for user `cefadc82cf2c4bac8ed508f9d26b2a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:17.344236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:17.394331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 930/1827] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:17.401731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:17.403956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:17.421755 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8303cd5-07f5-48fe-81b2-69354a3ecbff None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:17.428292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:17.429204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:17.429492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:17.429798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] PATH_INFO: `/v3/projects/9cf0d0db896b489784c6dcb15982a342` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:17.430984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf0d0db896b489784c6dcb15982a342)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:17.432746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 898/1828] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:17.433019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:17.433721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:17.438029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:17.500585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf0d0db896b489784c6dcb15982a342', 'request_id': 'req-5371e26c-f980-4dbd-962b-4b7245208767'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.509126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf0d0db896b489784c6dcb15982a342', 'request_id': 'req-5371e26c-f980-4dbd-962b-4b7245208767'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.515517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf187782-acef-435a-8b2d-c2248a1cab50 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:17.515959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf0d0db896b489784c6dcb15982a342', 'request_id': 'req-5371e26c-f980-4dbd-962b-4b7245208767'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.534927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] The token cache is being invalidate because project 9cf0d0db896b489784c6dcb15982a342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:17.535222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.535976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.537860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 931/1829] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:17 2022] DELETE /identity/v3/projects/9cf0d0db896b489784c6dcb15982a342 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:17.538880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:17.542288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:17.543936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:17.580874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f02246-1ee6-480b-beb7-77f50a189607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:17.581552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02246-1ee6-480b-beb7-77f50a189607 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:17.581830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02246-1ee6-480b-beb7-77f50a189607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:17.582113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02246-1ee6-480b-beb7-77f50a189607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:17.583920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-24f02246-1ee6-480b-beb7-77f50a189607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:17.607741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:17.625824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371e26c-f980-4dbd-962b-4b7245208767 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:17.633232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:17.635629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:17.636390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:17.636709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] PATH_INFO: `/v3/users/3483b26a20bb4e05badcc78195364b84` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:17.638139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3483b26a20bb4e05badcc78195364b84)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:17.640107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:17.659167 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f02246-1ee6-480b-beb7-77f50a189607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:17.660801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f02246-1ee6-480b-beb7-77f50a189607 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:17.679974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 899/1830] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:17.707474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3483b26a20bb4e05badcc78195364b84'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.708766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3483b26a20bb4e05badcc78195364b84'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.710237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3483b26a20bb4e05badcc78195364b84'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.723307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3483b26a20bb4e05badcc78195364b84'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.744978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3483b26a20bb4e05badcc78195364b84'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:17.762017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 932/1831] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:17 2022] DELETE /identity/v3/users/3483b26a20bb4e05badcc78195364b84 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:18.338460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc6817-07fd-405e-b26b-18d30eca0097 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:18.338935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc6817-07fd-405e-b26b-18d30eca0097 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:18.339076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc6817-07fd-405e-b26b-18d30eca0097 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:18.341066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 900/1832] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:59:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:59:18.386588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:18.389209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:18.413826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331c9b-2b51-4013-8f12-7e403675c0e2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:18.414726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331c9b-2b51-4013-8f12-7e403675c0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:18.415386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331c9b-2b51-4013-8f12-7e403675c0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:18.457500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f5331c9b-2b51-4013-8f12-7e403675c0e2 None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:18.462516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f5331c9b-2b51-4013-8f12-7e403675c0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:18.583264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab009f8e-efed-4b13-9efd-60a766454691 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:18.607933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:18.608902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:18.609179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:18.609449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] PATH_INFO: `/v3/projects/22d284b9f8c841828a73483f2efec06d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:18.611651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d284b9f8c841828a73483f2efec06d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:18.613917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:18.643944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 901/1833] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:59:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:18.645493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a698c1-7e60-411f-a574-c45e5d8905d7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:18.645790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a698c1-7e60-411f-a574-c45e5d8905d7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:18.646124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a698c1-7e60-411f-a574-c45e5d8905d7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:18.648043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 902/1834] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:59:18.648835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a698c1-7e60-411f-a574-c45e5d8905d7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:18.653259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-23a698c1-7e60-411f-a574-c45e5d8905d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:18.680971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a698c1-7e60-411f-a574-c45e5d8905d7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:18.689641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:18.693927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:18.695295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:18.695865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] PATH_INFO: `/v3/projects/3fc6083db1ff4d458c882d17f400087b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:18.699493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fc6083db1ff4d458c882d17f400087b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:18.702326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:18.744805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d284b9f8c841828a73483f2efec06d', 'request_id': 'req-c35dff29-4f4a-4329-afc0-f9e2e895c92d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.758020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d284b9f8c841828a73483f2efec06d', 'request_id': 'req-c35dff29-4f4a-4329-afc0-f9e2e895c92d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.770192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d284b9f8c841828a73483f2efec06d', 'request_id': 'req-c35dff29-4f4a-4329-afc0-f9e2e895c92d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.790820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] The token cache is being invalidate because project 22d284b9f8c841828a73483f2efec06d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:18.791082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.792193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c35dff29-4f4a-4329-afc0-f9e2e895c92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.797204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 933/1835] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:18 2022] DELETE /identity/v3/projects/22d284b9f8c841828a73483f2efec06d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:18.797204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:18.797204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:18.797204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:18.806595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:18.826011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:18.832678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:18.835990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fc6083db1ff4d458c882d17f400087b', 'request_id': 'req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.848013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fc6083db1ff4d458c882d17f400087b', 'request_id': 'req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.881150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fc6083db1ff4d458c882d17f400087b', 'request_id': 'req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.884231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:18.898358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] The token cache is being invalidate because project 3fc6083db1ff4d458c882d17f400087b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:18.898358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.899437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:18.906372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 903/1836] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:18 2022] DELETE /identity/v3/projects/3fc6083db1ff4d458c882d17f400087b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:18.906372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:18.907267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:18.956051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 934/1837] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:59:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:18.957245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:18.959350 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:19.057809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9100649-8923-4b0f-b3fc-396c7ccf5c4f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.065625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ba627-6d53-4314-88e9-8f0f6d8c361a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.066173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef1214-d535-4c32-b7c3-ae0f2eecd9a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.067165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef1214-d535-4c32-b7c3-ae0f2eecd9a0 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.069290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef1214-d535-4c32-b7c3-ae0f2eecd9a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.070092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef1214-d535-4c32-b7c3-ae0f2eecd9a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.072934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.073828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.075270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.075270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] PATH_INFO: `/v3/users/f90a5e161a3a4978829ee4e261ae60de` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.075838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90a5e161a3a4978829ee4e261ae60de)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.076468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a6ef1214-d535-4c32-b7c3-ae0f2eecd9a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:19.078097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.107428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef1214-d535-4c32-b7c3-ae0f2eecd9a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.112062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef1214-d535-4c32-b7c3-ae0f2eecd9a0 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.132541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 904/1838] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:19.138487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.138796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.139225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.180974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:19.182361 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:19.195986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90a5e161a3a4978829ee4e261ae60de'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:19.197789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90a5e161a3a4978829ee4e261ae60de'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:19.211601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 905/1839] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:19.212785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:19.218325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90a5e161a3a4978829ee4e261ae60de'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:19.221233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:19.233325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90a5e161a3a4978829ee4e261ae60de'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:19.245325 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7af210-5e98-4ec7-aa47-13293de093a6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.254004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.254865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.255149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.255425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.258174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:19.269699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-658f2a72-6c7c-4102-a9d2-f69e45833823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90a5e161a3a4978829ee4e261ae60de'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:19.274642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.277282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.296567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 935/1840] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:18 2022] DELETE /identity/v3/users/f90a5e161a3a4978829ee4e261ae60de => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:19.298190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c8c90-bbcb-43ba-9655-ea62b4c8bbff None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.299435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c8c90-bbcb-43ba-9655-ea62b4c8bbff None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.299881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c8c90-bbcb-43ba-9655-ea62b4c8bbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.309676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 906/1841] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:19.375035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d26c8c90-bbcb-43ba-9655-ea62b4c8bbff None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:19.379241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d26c8c90-bbcb-43ba-9655-ea62b4c8bbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:19.424790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:19.616145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 936/1842] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:19.617628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:19.632303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332ad8-3fd3-4ca6-97af-4f39731d09a6 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.638693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.639644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.639979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.640260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.641458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:19.642617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.645598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.673707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 907/1843] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:19.678610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c8c90-bbcb-43ba-9655-ea62b4c8bbff None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:19.691073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c8c90-bbcb-43ba-9655-ea62b4c8bbff None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.696017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.696878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.697164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.697436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.698132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.700893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.737224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 937/1844] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:19 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:19.742029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:19.755570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d82db1-3208-42e6-9968-1246f15c981f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.828251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.828974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.829259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.829564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.830519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.832532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.859880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 908/1845] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:19 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:19.862476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:19.876842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bfa6e-7caa-4600-865a-41fc11fdf410 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.891775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.891775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.891775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.891775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.892118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.895059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.911314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 938/1846] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:19.912288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:19.929362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e70428-edac-4e87-8b5f-e995682dbd20 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.938578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fb76d-afd6-46ff-823c-408231137ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.939683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fb76d-afd6-46ff-823c-408231137ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.940004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fb76d-afd6-46ff-823c-408231137ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.940691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fb76d-afd6-46ff-823c-408231137ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:19.941511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fb76d-afd6-46ff-823c-408231137ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:19.943364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fb76d-afd6-46ff-823c-408231137ca2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:19.958830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 909/1847] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:19.962978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:19.975195 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de8af6-bde4-41b4-8736-8fd31c296386 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:19.981485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:19.982430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:19.982712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:19.982988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] PATH_INFO: `/v3/projects/906acca2f4504573a34a824f06214500/users/34822db16e1c45f0833da8d930ea89c7/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.010812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] RBAC: Authorizing `identity:create_grant(project_id=906acca2f4504573a34a824f06214500, user_id=34822db16e1c45f0833da8d930ea89c7, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:20.014689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:20.036602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 939/1848] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:19 2022] PUT /identity/v3/projects/906acca2f4504573a34a824f06214500/users/34822db16e1c45f0833da8d930ea89c7/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:20.049007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a94f-60fa-42cf-8fa8-efe375e82305 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.049286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a94f-60fa-42cf-8fa8-efe375e82305 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.049851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a94f-60fa-42cf-8fa8-efe375e82305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.084228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1656a94f-60fa-42cf-8fa8-efe375e82305 None None] MFA Rules not processed for user `34822db16e1c45f0833da8d930ea89c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:20.085738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1656a94f-60fa-42cf-8fa8-efe375e82305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:20.101197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:20.228233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 910/1849] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:20.233235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:20.350350 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8646aa-6ee5-476f-82af-6f70ea7fffca admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:20.355998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6972d3-3546-44a7-a563-ea292c082885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:20.356953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6972d3-3546-44a7-a563-ea292c082885 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.357244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6972d3-3546-44a7-a563-ea292c082885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.357523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6972d3-3546-44a7-a563-ea292c082885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.358588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6972d3-3546-44a7-a563-ea292c082885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:20.360580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6972d3-3546-44a7-a563-ea292c082885 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:20.393170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 940/1850] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:20 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:20.399337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656a94f-60fa-42cf-8fa8-efe375e82305 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:20.412027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656a94f-60fa-42cf-8fa8-efe375e82305 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:20.448179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8b0b-acbf-4ca5-b021-6ef0ed2e92bc None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.450788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8b0b-acbf-4ca5-b021-6ef0ed2e92bc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.452636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8b0b-acbf-4ca5-b021-6ef0ed2e92bc None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.455225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 941/1851] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:59:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 15:59:20.486527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ecce3-4fb0-42f0-aadf-95075e2be167 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.486830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ecce3-4fb0-42f0-aadf-95075e2be167 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.487229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ecce3-4fb0-42f0-aadf-95075e2be167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.507317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:20.508252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.508681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.509052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.509942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:20.512152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:20.535693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-315ecce3-4fb0-42f0-aadf-95075e2be167 None None] MFA Rules not processed for user `0372966e33f6413ebe529c001bbb2518`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:20.537293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-315ecce3-4fb0-42f0-aadf-95075e2be167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:20.550749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 911/1852] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:20 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:20.555600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:20.559909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:20.585084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cafdba-148c-45ff-8480-63446afd11d2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:20.591784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:20.592969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.593246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.593848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] PATH_INFO: `/v3/users/adb6ac0134cf4740b03db33cfaeb4457` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.595680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb6ac0134cf4740b03db33cfaeb4457)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:20.598025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:20.620059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:20.678465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb6ac0134cf4740b03db33cfaeb4457'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:20.679151 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb6ac0134cf4740b03db33cfaeb4457'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:20.690255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 942/1853] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:20.695489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ecce3-4fb0-42f0-aadf-95075e2be167 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:20.702759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb6ac0134cf4740b03db33cfaeb4457'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:20.703499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb6ac0134cf4740b03db33cfaeb4457'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:20.726298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb6ac0134cf4740b03db33cfaeb4457'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:20.730578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ecce3-4fb0-42f0-aadf-95075e2be167 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:20.745632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:20.746832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.747125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.747411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.749545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:20.753990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:20.766367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 912/1854] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:20 2022] DELETE /identity/v3/users/adb6ac0134cf4740b03db33cfaeb4457 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:20.766367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:20.768834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:20.776850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 943/1855] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:20.780875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:20.782855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:20.798584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c5225-c506-4147-a6f8-87832a0db02a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:20.807625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:20.885799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:20.887180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.887464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.887913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] PATH_INFO: `/v3/projects/d05de4c7c8604ce389fbe663ca051dd7/users/fb3e3b4a79654cd6a4eb0115cbb58492/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.909420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05de4c7c8604ce389fbe663ca051dd7, user_id=fb3e3b4a79654cd6a4eb0115cbb58492, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:20.911931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c12ce-e1b5-4bd1-9bb7-73e9683b02c1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:20.912301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:20.919565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:20.921309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:20.921636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:20.921949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:20.925742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:20.948057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:20.949862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:20.952505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 944/1856] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:20 2022] PUT /identity/v3/projects/d05de4c7c8604ce389fbe663ca051dd7/users/fb3e3b4a79654cd6a4eb0115cbb58492/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:20.966665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:20.968230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 913/1857] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:21.041279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:21.042795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:21.065915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9551bd9-201d-4466-8687-915cdda24c45 service placement] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:21.068933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8aa0e-89eb-4fe9-b147-8a4091c32ee6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:21.075423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:21.076403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.076624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:21.076862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.077249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.078070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:21.079089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.079536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.080697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:21.081260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] PATH_INFO: `/v3/projects/2cea31c70e1a40c790e4d1dcd6b7410d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.083174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cea31c70e1a40c790e4d1dcd6b7410d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:21.085725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:21.101991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 945/1858] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:21.103018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:21.115498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb359ece-5b2a-462b-8178-b5e8e3df68f4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:21.120923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:21.122338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.122659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.122945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] PATH_INFO: `/v3/projects/d05de4c7c8604ce389fbe663ca051dd7/users/fb3e3b4a79654cd6a4eb0115cbb58492/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.125348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05de4c7c8604ce389fbe663ca051dd7, user_id=fb3e3b4a79654cd6a4eb0115cbb58492, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:21.129329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:21.142109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 946/1859] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:21 2022] PUT /identity/v3/projects/d05de4c7c8604ce389fbe663ca051dd7/users/fb3e3b4a79654cd6a4eb0115cbb58492/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:21.143089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:21.149393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:21.185874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cea31c70e1a40c790e4d1dcd6b7410d', 'request_id': 'req-02dbee34-3266-40c0-abe7-37751cd14b3b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.193338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cea31c70e1a40c790e4d1dcd6b7410d', 'request_id': 'req-02dbee34-3266-40c0-abe7-37751cd14b3b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.198705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:21.203898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cea31c70e1a40c790e4d1dcd6b7410d', 'request_id': 'req-02dbee34-3266-40c0-abe7-37751cd14b3b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.220236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] The token cache is being invalidate because project 2cea31c70e1a40c790e4d1dcd6b7410d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:21.220562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.221445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02dbee34-3266-40c0-abe7-37751cd14b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.223559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 914/1860] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:21 2022] DELETE /identity/v3/projects/2cea31c70e1a40c790e4d1dcd6b7410d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:21.224833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.225124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.225406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.239375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019395f-e4a7-4ec2-9a9b-59eb0358375f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:21.262147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] MFA Rules not processed for user `fb3e3b4a79654cd6a4eb0115cbb58492`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:21.263858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:21.327902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:21.328916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.329368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.329883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] PATH_INFO: `/v3/projects/2e1c9c1858334efe85bdec9da1ade35f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.331222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1c9c1858334efe85bdec9da1ade35f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:21.335300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:21.357425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 915/1861] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:21.376152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:21.380821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:21.410424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1c9c1858334efe85bdec9da1ade35f', 'request_id': 'req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.416114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dab76-8cf5-401e-9a90-f64ff6881cc6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:21.416876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1c9c1858334efe85bdec9da1ade35f', 'request_id': 'req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.424388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1c9c1858334efe85bdec9da1ade35f', 'request_id': 'req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.440232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:21.456711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] The token cache is being invalidate because project 2e1c9c1858334efe85bdec9da1ade35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:21.457189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.457798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.459877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 947/1862] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:21 2022] DELETE /identity/v3/projects/2e1c9c1858334efe85bdec9da1ade35f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:21.460943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:21.463452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:21.540665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:21.541746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.542036 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.542457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] PATH_INFO: `/v3/users/cefadc82cf2c4bac8ed508f9d26b2a27` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.543736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cefadc82cf2c4bac8ed508f9d26b2a27)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:21.545723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:21.597095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439d11b-e8ee-4ba8-b7f1-5f1d4424976a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:21.606213 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:21.607441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.607954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.608557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.612085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:21.647628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cefadc82cf2c4bac8ed508f9d26b2a27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.648941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cefadc82cf2c4bac8ed508f9d26b2a27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.660777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cefadc82cf2c4bac8ed508f9d26b2a27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.661525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cefadc82cf2c4bac8ed508f9d26b2a27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.664740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:21.673104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fdbfac54-8a89-4af3-b2bc-5f46ebf2aa54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cefadc82cf2c4bac8ed508f9d26b2a27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:21.737908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 916/1863] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:21 2022] DELETE /identity/v3/users/cefadc82cf2c4bac8ed508f9d26b2a27 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:21.755200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1d1d-3a71-4be5-948e-5a8aaa3b923a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:21.756273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1d1d-3a71-4be5-948e-5a8aaa3b923a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:21.757848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1d1d-3a71-4be5-948e-5a8aaa3b923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:21.758487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:21.762354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:21.807321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 948/1864] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:21 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:21.822131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-44bb1d1d-3a71-4be5-948e-5a8aaa3b923a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:21.826333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-44bb1d1d-3a71-4be5-948e-5a8aaa3b923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:21.883758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:21.896958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb9906-f59c-414e-9e24-d88e347912e5 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:21.998376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 917/1865] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:22.009645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b50acc-e4c5-443b-8c08-22da1ede9426 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.009942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b50acc-e4c5-443b-8c08-22da1ede9426 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.010681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b50acc-e4c5-443b-8c08-22da1ede9426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.017845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.018578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.018864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.019252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.021174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:22.046881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-75b50acc-e4c5-443b-8c08-22da1ede9426 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:22.050222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-75b50acc-e4c5-443b-8c08-22da1ede9426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:22.065634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 918/1866] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:22.070502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b50acc-e4c5-443b-8c08-22da1ede9426 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.082260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b50acc-e4c5-443b-8c08-22da1ede9426 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.087482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.088444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.088922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.089259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.090045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:22.090436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.092037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.104907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.106992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.107101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 919/1867] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:59:22.111759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.114521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:22.118919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 949/1868] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:22.119725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.130580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0ac5-665a-4d5e-a2cb-64ccc63e49eb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.131796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b600-939c-4881-b4ef-5c572becb692 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.137801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.138676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.138959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.139233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] PATH_INFO: `/v3/projects/3641a2c142c74c25aa8212c7ee592d5f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.141476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3641a2c142c74c25aa8212c7ee592d5f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.144248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.150026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.150026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.150026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.150026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.150391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.150391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.155345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:22.186141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 950/1869] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:22.191556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.216305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3641a2c142c74c25aa8212c7ee592d5f', 'request_id': 'req-9673850a-6b95-4fff-8c63-5b9230d574ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:22.226135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3641a2c142c74c25aa8212c7ee592d5f', 'request_id': 'req-9673850a-6b95-4fff-8c63-5b9230d574ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:22.237163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3641a2c142c74c25aa8212c7ee592d5f', 'request_id': 'req-9673850a-6b95-4fff-8c63-5b9230d574ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:22.252628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] The token cache is being invalidate because project 3641a2c142c74c25aa8212c7ee592d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:22.252970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:22.254300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9673850a-6b95-4fff-8c63-5b9230d574ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:22.259145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 920/1870] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:22 2022] DELETE /identity/v3/projects/3641a2c142c74c25aa8212c7ee592d5f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:22.269040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651e5fc-b94a-45aa-9f4b-831cd3784bc0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.269902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651e5fc-b94a-45aa-9f4b-831cd3784bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.270291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651e5fc-b94a-45aa-9f4b-831cd3784bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.286239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:22.296930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4651e5fc-b94a-45aa-9f4b-831cd3784bc0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:22.299989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4651e5fc-b94a-45aa-9f4b-831cd3784bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:22.304334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643bdf-5563-4ea7-a385-e369a90a2d66 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.401079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 921/1871] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:22.409902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dff500-8bc0-467f-8200-036018f55fcf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.410227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dff500-8bc0-467f-8200-036018f55fcf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.411224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dff500-8bc0-467f-8200-036018f55fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.440096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.442024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.443465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.443998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.445221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.448369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.482949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 951/1872] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/users => generated 291 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:22.487159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.491337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:22.494889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b4dff500-8bc0-467f-8200-036018f55fcf None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:22.497442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b4dff500-8bc0-467f-8200-036018f55fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:22.528645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b2e03-a19d-4460-a121-4ef71799ebb2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.537336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.538310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.538668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.538990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.540017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.542199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.555503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 922/1873] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:22.560848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dff500-8bc0-467f-8200-036018f55fcf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.587053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 952/1874] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:22.593143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dff500-8bc0-467f-8200-036018f55fcf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.594339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.619217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2d2fc-7b88-4091-a715-46bcf02ff7d8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.622108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.623788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.624225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.624686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.625893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:22.627017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.630063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.633840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.636054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.636855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.650925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 923/1875] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:59:22.657021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.659087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.660170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.662225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.679297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0abbf-931e-4edd-ac82-9a4dde61f6ac admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.692583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.693412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.693695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.693962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.695358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.697210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.711789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 953/1876] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:22.716555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.738364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d28295-b115-4945-a105-522ce0199136 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.765934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c65e2-bb2e-4b35-8e9e-eb18057d4235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:22.767712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c65e2-bb2e-4b35-8e9e-eb18057d4235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.768937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c65e2-bb2e-4b35-8e9e-eb18057d4235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.770162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c65e2-bb2e-4b35-8e9e-eb18057d4235 admin admin] PATH_INFO: `/v3/projects/fb1726933499469aa6f24aca900d9768/users/b89306dca04e4ec9992045ab50b65da6/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.774562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 924/1877] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:22.780067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:22.805258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c65e2-bb2e-4b35-8e9e-eb18057d4235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1726933499469aa6f24aca900d9768, user_id=b89306dca04e4ec9992045ab50b65da6, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:22.808965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c65e2-bb2e-4b35-8e9e-eb18057d4235 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:22.835961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 954/1878] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:22 2022] PUT /identity/v3/projects/fb1726933499469aa6f24aca900d9768/users/b89306dca04e4ec9992045ab50b65da6/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:22.838962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a82a4-631f-40fd-a250-78bc45e47b1b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:22.839269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a82a4-631f-40fd-a250-78bc45e47b1b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:22.839545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a82a4-631f-40fd-a250-78bc45e47b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:22.898189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:22.912690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acbb7d-83c9-4220-ad50-796f9acf4cf1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:22.913746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-570a82a4-631f-40fd-a250-78bc45e47b1b None None] MFA Rules not processed for user `b89306dca04e4ec9992045ab50b65da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:22.915133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-570a82a4-631f-40fd-a250-78bc45e47b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:23.042901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 955/1879] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:23.047636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a82a4-631f-40fd-a250-78bc45e47b1b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.067136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.068166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.068550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.068902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.069675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.072471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.090942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:23.098817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 925/1880] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:22 2022] POST /identity/v3/users => generated 291 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:23.102762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.116600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e365-657f-4550-b2f0-91502ddf2b98 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.122877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.123564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.123847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.124121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.125155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.133873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.138200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a82a4-631f-40fd-a250-78bc45e47b1b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.143656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 926/1881] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:23.144157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.145052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.145340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.145617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.146377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.147691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.154932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.169643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd404fc-6856-4cac-ba9b-3c1e7effbf18 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.179540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.180586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.180937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.181280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.183187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.185684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.192611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 956/1882] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:23 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:23.196513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.203177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 927/1883] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:23.207706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.222062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56daa52-277f-4f17-b699-aa698b26bd33 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.303279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94122b03-ecf7-44ea-9913-d13146da6daa admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.310544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d11497-2aa4-468e-8e29-37fb6f55186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.310852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d11497-2aa4-468e-8e29-37fb6f55186a admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.311691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d11497-2aa4-468e-8e29-37fb6f55186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.312137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d11497-2aa4-468e-8e29-37fb6f55186a admin admin] PATH_INFO: `/v3/projects/5233e79acf774ccda021282775c90e64/users/25a9500348074250aea0a517f8139938/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.316895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.317650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.317939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.318215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.318918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.321173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.344325 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d11497-2aa4-468e-8e29-37fb6f55186a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5233e79acf774ccda021282775c90e64, user_id=25a9500348074250aea0a517f8139938, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.348266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d11497-2aa4-468e-8e29-37fb6f55186a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.350292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 957/1884] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:23 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:23.353458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.365176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5bc07-83d6-4d11-93ad-1fbfa7dd7890 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.371508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 928/1885] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:23 2022] PUT /identity/v3/projects/5233e79acf774ccda021282775c90e64/users/25a9500348074250aea0a517f8139938/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:23.382558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fced7f3-4029-48a1-8624-f4c9fc7adf6f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.382855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fced7f3-4029-48a1-8624-f4c9fc7adf6f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.383305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fced7f3-4029-48a1-8624-f4c9fc7adf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.430218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4fced7f3-4029-48a1-8624-f4c9fc7adf6f None None] MFA Rules not processed for user `25a9500348074250aea0a517f8139938`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:23.435291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4fced7f3-4029-48a1-8624-f4c9fc7adf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:23.457548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.458243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.458537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.458818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.459547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.462056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.476403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 958/1886] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:23.479470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.506030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb1da7-bce2-46fc-a173-60674349da24 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.509373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:23.512729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.513923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.514385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.514786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] PATH_INFO: `/v3/projects/45856fbd40f444daa435123bba226363/users/827aad79f49645298aa84bf7b1f4aab5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.542165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 929/1887] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:23.545582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45856fbd40f444daa435123bba226363, user_id=827aad79f49645298aa84bf7b1f4aab5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.545582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.546664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fced7f3-4029-48a1-8624-f4c9fc7adf6f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.565621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fced7f3-4029-48a1-8624-f4c9fc7adf6f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.570950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 959/1888] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:23 2022] PUT /identity/v3/projects/45856fbd40f444daa435123bba226363/users/827aad79f49645298aa84bf7b1f4aab5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:23.573351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.630788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:23.658596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.659260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.659541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.659803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.660779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.662416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.681296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76229ae7-4977-46e0-aff1-7179fafb3837 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.687998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.689636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.690017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.690366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.691276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.694087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.697922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 930/1889] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:23 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:23.701906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.733930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 960/1890] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:23.737579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.765765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3e578-a175-4a9e-96ac-eb55b41b18b4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.790195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c299-07d8-4fce-9096-b494abc8419f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.802240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.803588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.803878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.804159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.804896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.807109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.836595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 931/1891] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:23 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:23.837722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.852817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a052ad5-5682-4d4b-b662-77dea7c0a4cd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.856776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ceacc-9f13-4e05-bfa8-5f41788f0c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.857763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ceacc-9f13-4e05-bfa8-5f41788f0c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.858049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ceacc-9f13-4e05-bfa8-5f41788f0c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.858394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ceacc-9f13-4e05-bfa8-5f41788f0c0b admin admin] PATH_INFO: `/v3/projects/45856fbd40f444daa435123bba226363/users/827aad79f49645298aa84bf7b1f4aab5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.859831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.860469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.860832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.861149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.861357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ceacc-9f13-4e05-bfa8-5f41788f0c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45856fbd40f444daa435123bba226363, user_id=827aad79f49645298aa84bf7b1f4aab5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.862010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:23.863792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.873712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ceacc-9f13-4e05-bfa8-5f41788f0c0b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:23.877830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 932/1892] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:23.886496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:23.894963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 961/1893] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:23 2022] PUT /identity/v3/projects/45856fbd40f444daa435123bba226363/users/827aad79f49645298aa84bf7b1f4aab5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:23.899732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827dfde-6cd3-4875-87c0-a16bf6901ee6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:23.907034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.908774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.909554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:23.958532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] MFA Rules not processed for user `827aad79f49645298aa84bf7b1f4aab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:23.959908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:23.962391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:23.990517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:23.991787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:23.992377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:23.992809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] PATH_INFO: `/v3/projects/6d42a3e16dcd412d8058abe58ac159c4/users/c94dfcbfcb114d4caef320b88586a407/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:24.015857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d42a3e16dcd412d8058abe58ac159c4, user_id=c94dfcbfcb114d4caef320b88586a407, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:24.019593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:24.040397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 933/1894] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:23 2022] PUT /identity/v3/projects/6d42a3e16dcd412d8058abe58ac159c4/users/c94dfcbfcb114d4caef320b88586a407/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:24.044538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:24.051359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 962/1895] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:24.071070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:24.072700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:24.138497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:24.178035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050d14b-09b4-46a3-9fc4-603f7e1eaf63 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:24.180915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd3b48-8cbc-4de0-892b-96e809e9c1e5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:24.185393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:24.186301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:24.186582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:24.187311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:24.187764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:24.191197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:24.205068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 934/1896] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:24.209951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:24.225220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2762335-5682-4de3-ba57-3e2947edaef1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:24.233393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfb065-8f26-4f11-8baf-c06e7627bfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:24.234586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfb065-8f26-4f11-8baf-c06e7627bfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:24.234586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfb065-8f26-4f11-8baf-c06e7627bfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:24.234983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfb065-8f26-4f11-8baf-c06e7627bfa8 admin admin] PATH_INFO: `/v3/projects/6d42a3e16dcd412d8058abe58ac159c4/users/c94dfcbfcb114d4caef320b88586a407/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:24.238843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfb065-8f26-4f11-8baf-c06e7627bfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d42a3e16dcd412d8058abe58ac159c4, user_id=c94dfcbfcb114d4caef320b88586a407, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:24.243496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfb065-8f26-4f11-8baf-c06e7627bfa8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:24.252824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:24.254065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:24.254687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:24.255164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:24.264938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 935/1897] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:24 2022] PUT /identity/v3/projects/6d42a3e16dcd412d8058abe58ac159c4/users/c94dfcbfcb114d4caef320b88586a407/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:24.277420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:24.277897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:24.278338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:24.331950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:24.333466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:24.347406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] MFA Rules not processed for user `c94dfcbfcb114d4caef320b88586a407`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:24.350608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:24.355617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 963/1898] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:24.480967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 936/1899] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:24.492945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:24.503669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421db45-a354-47ad-89d2-12a44233fa16 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:24.573821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:24.574874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:24.574984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:24.575269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:24.589189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:24.591093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:24.601926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 964/1900] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:24.857727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:24.859129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:24.882480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a682b8-bd95-4c24-9105-e351497252e2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:24.894939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71fbaa-e4b6-42e4-b8cb-c0e1762722d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:24.895810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71fbaa-e4b6-42e4-b8cb-c0e1762722d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:24.896203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71fbaa-e4b6-42e4-b8cb-c0e1762722d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:24.897491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71fbaa-e4b6-42e4-b8cb-c0e1762722d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:24.973722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:25.015448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71fbaa-e4b6-42e4-b8cb-c0e1762722d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:25.017086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71fbaa-e4b6-42e4-b8cb-c0e1762722d7 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:25.029373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 937/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:25.449307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:25.594075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:25.628613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0651cc4-0f15-4785-b144-d68df175a8ec service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:25.635035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:25.636829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:25.637224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:25.637605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] PATH_INFO: `/v3/users/b89306dca04e4ec9992045ab50b65da6` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:25.639252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89306dca04e4ec9992045ab50b65da6)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:25.643313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:25.734766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89306dca04e4ec9992045ab50b65da6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:25.735564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89306dca04e4ec9992045ab50b65da6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:25.736775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89306dca04e4ec9992045ab50b65da6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:25.749367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89306dca04e4ec9992045ab50b65da6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:25.768124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89306dca04e4ec9992045ab50b65da6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:25.783699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 965/1902] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:25 2022] DELETE /identity/v3/users/b89306dca04e4ec9992045ab50b65da6 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:25.787998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:25.788284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:25.788597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:25.807505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:25.808904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:25.892534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 938/1903] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:25.898848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:25.899398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:25.987133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3913d-4366-4058-b3fb-8f0adab99418 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:25.993150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:25.993875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:25.994241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:25.994576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:26.012739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:26.015561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:26.033285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 966/1904] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:26.489903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:26.494979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:26.523847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:26.525770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:26.536623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2102565-9033-4c77-893c-e21ef696f4cb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:26.541705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f9d6-3f55-400f-8f86-3ccc97074f87 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:26.542354 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:26.544687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:26.545040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:26.545877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] PATH_INFO: `/v3/users/34822db16e1c45f0833da8d930ea89c7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:26.546790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34822db16e1c45f0833da8d930ea89c7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:26.549987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:26.555150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:26.556260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:26.556568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:26.557266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] PATH_INFO: `/v3/projects/fb1726933499469aa6f24aca900d9768` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:26.558707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1726933499469aa6f24aca900d9768)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:26.561317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:26.609684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34822db16e1c45f0833da8d930ea89c7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.610445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34822db16e1c45f0833da8d930ea89c7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.619524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34822db16e1c45f0833da8d930ea89c7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.621873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34822db16e1c45f0833da8d930ea89c7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.631605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af918ac6-e657-4ba2-a9cd-d2c5b4237ba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34822db16e1c45f0833da8d930ea89c7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.648025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1726933499469aa6f24aca900d9768', 'request_id': 'req-29a6ac99-8685-45a9-97f5-e18ffcba16be'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.656503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1726933499469aa6f24aca900d9768', 'request_id': 'req-29a6ac99-8685-45a9-97f5-e18ffcba16be'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.658447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 939/1905] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:26 2022] DELETE /identity/v3/users/34822db16e1c45f0833da8d930ea89c7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:26.660794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:26.662870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:26.663299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:26.665660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:26.669054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1726933499469aa6f24aca900d9768', 'request_id': 'req-29a6ac99-8685-45a9-97f5-e18ffcba16be'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.685215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:26.686527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:26.690366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] The token cache is being invalidate because project fb1726933499469aa6f24aca900d9768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:26.690983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.691815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.694682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 967/1906] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:26 2022] DELETE /identity/v3/projects/fb1726933499469aa6f24aca900d9768 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:26.698304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:26.699786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:26.748920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:26.774030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 940/1907] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:26.780666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:26.782136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:26.800166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6ac99-8685-45a9-97f5-e18ffcba16be admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:26.815070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:26.815908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:26.816267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:26.816642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] PATH_INFO: `/v3/users/827aad79f49645298aa84bf7b1f4aab5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:26.820684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] RBAC: Authorizing `identity:delete_user(user_id=827aad79f49645298aa84bf7b1f4aab5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:26.822991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:26.915456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b10d2c-620c-4217-a757-55d5b4c9e2d3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:26.919798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '827aad79f49645298aa84bf7b1f4aab5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.921430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '827aad79f49645298aa84bf7b1f4aab5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.924100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '827aad79f49645298aa84bf7b1f4aab5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.928733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:26.930014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:26.930446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:26.931212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:26.933776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:26.936628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '827aad79f49645298aa84bf7b1f4aab5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.950855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '827aad79f49645298aa84bf7b1f4aab5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:26.951012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:26.952511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:26.964554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 941/1908] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:26.970228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 968/1909] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:26 2022] DELETE /identity/v3/users/827aad79f49645298aa84bf7b1f4aab5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:27.282581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.284257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:27.297316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3c48f-07f3-4411-9a01-360af0671e83 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.371065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.372144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.372515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.372805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] PATH_INFO: `/v3/projects/906acca2f4504573a34a824f06214500` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.374222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=906acca2f4504573a34a824f06214500)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.377637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.379097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.381150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:27.395213 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80325c3-e363-4d11-a74a-d806d925819e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.400116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.401081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.401431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.401791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] PATH_INFO: `/v3/projects/45856fbd40f444daa435123bba226363` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.403498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=45856fbd40f444daa435123bba226363)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.405642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.461063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45856fbd40f444daa435123bba226363', 'request_id': 'req-26860b9d-19eb-456b-90d3-8bde3c56c9db'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.463727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '906acca2f4504573a34a824f06214500', 'request_id': 'req-ec323b72-169b-4c84-8bb8-9d1727037a13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.466844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45856fbd40f444daa435123bba226363', 'request_id': 'req-26860b9d-19eb-456b-90d3-8bde3c56c9db'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.470735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '906acca2f4504573a34a824f06214500', 'request_id': 'req-ec323b72-169b-4c84-8bb8-9d1727037a13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.474233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45856fbd40f444daa435123bba226363', 'request_id': 'req-26860b9d-19eb-456b-90d3-8bde3c56c9db'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.476124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:27.479510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '906acca2f4504573a34a824f06214500', 'request_id': 'req-ec323b72-169b-4c84-8bb8-9d1727037a13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.491628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] The token cache is being invalidate because project 906acca2f4504573a34a824f06214500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:27.491910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.492029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] The token cache is being invalidate because project 45856fbd40f444daa435123bba226363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:27.492320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.492560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.493213 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-26860b9d-19eb-456b-90d3-8bde3c56c9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.494188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 942/1910] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:27 2022] DELETE /identity/v3/projects/906acca2f4504573a34a824f06214500 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:27.495484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 969/1911] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:27 2022] DELETE /identity/v3/projects/45856fbd40f444daa435123bba226363 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:27.498216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.499708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:27.499963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113074-4ebb-4aaf-b542-4042dcf29c08 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.500262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113074-4ebb-4aaf-b542-4042dcf29c08 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.500552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113074-4ebb-4aaf-b542-4042dcf29c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.519474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-51113074-4ebb-4aaf-b542-4042dcf29c08 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:27.520852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-51113074-4ebb-4aaf-b542-4042dcf29c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:27.525256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:27.567458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec323b72-169b-4c84-8bb8-9d1727037a13 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.584108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 970/1912] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:27.591788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a4879-4c44-432a-87cf-e56269728415 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.592142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a4879-4c44-432a-87cf-e56269728415 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.592775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a4879-4c44-432a-87cf-e56269728415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.614860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-115a4879-4c44-432a-87cf-e56269728415 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:27.616141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-115a4879-4c44-432a-87cf-e56269728415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:27.629498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 971/1913] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:27.633629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a4879-4c44-432a-87cf-e56269728415 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.633854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.635399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.635658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.635902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] PATH_INFO: `/v3/users/fb3e3b4a79654cd6a4eb0115cbb58492` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.637193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3e3b4a79654cd6a4eb0115cbb58492)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.639066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.644084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a4879-4c44-432a-87cf-e56269728415 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.648904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.650173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.650484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.650758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.651479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:27.651780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.653425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.666478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 972/1914] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:27.671178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.684374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbbd87-09b8-4646-88cf-9de529c86512 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.690514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.691268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.691561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.691832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.692564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.694464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.699043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3e3b4a79654cd6a4eb0115cbb58492'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.699793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3e3b4a79654cd6a4eb0115cbb58492'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.708045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3e3b4a79654cd6a4eb0115cbb58492'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.708773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3e3b4a79654cd6a4eb0115cbb58492'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.717677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3e3b4a79654cd6a4eb0115cbb58492'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:27.727842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 973/1915] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:27 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:27.731967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.733900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:27.744214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 943/1916] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:27 2022] DELETE /identity/v3/users/fb3e3b4a79654cd6a4eb0115cbb58492 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:27.804288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cff2a3-3a59-4571-acc7-b2343f4c1057 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.859321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:27.867807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.868987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.869557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.869963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.870738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.873039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.893710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 974/1917] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:27 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:27.898100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.916390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b73261-42a8-42eb-97c6-a4062a2c4c23 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.930888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.931715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.931997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.933001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.933347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.934965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.946149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 944/1918] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:27.950132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:27.961124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77463d1-a23b-4369-837c-eaa7b2489e19 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:27.966080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95311858-627d-4e37-9315-fa77504d5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:27.966743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311858-627d-4e37-9315-fa77504d5882 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:27.967031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311858-627d-4e37-9315-fa77504d5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:27.967470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311858-627d-4e37-9315-fa77504d5882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:27.968067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95311858-627d-4e37-9315-fa77504d5882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:27.969826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95311858-627d-4e37-9315-fa77504d5882 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:27.989580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 975/1919] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:27.993599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.004262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930c12e-b782-4aa1-9daa-0349a5823c56 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.010059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.011219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.011661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.011942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] PATH_INFO: `/v3/projects/c3e239184fd748e09c59e5e031ce3e06/users/82c5ed38278e4b2ab8a6a0f1a6272ba6/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.029398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e239184fd748e09c59e5e031ce3e06, user_id=82c5ed38278e4b2ab8a6a0f1a6272ba6, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.032242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.051973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 945/1920] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:27 2022] PUT /identity/v3/projects/c3e239184fd748e09c59e5e031ce3e06/users/82c5ed38278e4b2ab8a6a0f1a6272ba6/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:28.063125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.063572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.064294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.097192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] MFA Rules not processed for user `82c5ed38278e4b2ab8a6a0f1a6272ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:28.098602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:28.195375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 976/1921] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:28.200238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.219452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:28.318109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5514a5-16fb-4814-b337-cdc3017e35ab admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.334747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.335789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.336216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.336691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.337942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.341560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.370412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.371923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:28.420302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951187b-945a-4b42-af75-7581870e5d2b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.430700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 946/1922] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:28 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:28.434750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.441634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.451667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.453650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.455221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] PATH_INFO: `/v3/projects/d05de4c7c8604ce389fbe663ca051dd7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.459833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05de4c7c8604ce389fbe663ca051dd7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.463020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.559706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fef3d-80d3-4c62-84c0-c5abccc7f655 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.584653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05de4c7c8604ce389fbe663ca051dd7', 'request_id': 'req-3dfbc693-0018-4996-96d8-dc4e4ba475e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:28.600858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05de4c7c8604ce389fbe663ca051dd7', 'request_id': 'req-3dfbc693-0018-4996-96d8-dc4e4ba475e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:28.620624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05de4c7c8604ce389fbe663ca051dd7', 'request_id': 'req-3dfbc693-0018-4996-96d8-dc4e4ba475e7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:28.672713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] The token cache is being invalidate because project d05de4c7c8604ce389fbe663ca051dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:28.676797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:28.679731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3dfbc693-0018-4996-96d8-dc4e4ba475e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:28.681980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 977/1923] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:28 2022] DELETE /identity/v3/projects/d05de4c7c8604ce389fbe663ca051dd7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:28.686933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d765858-b913-404e-9ebc-c0c2431c6b72 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.687337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d765858-b913-404e-9ebc-c0c2431c6b72 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.687716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d765858-b913-404e-9ebc-c0c2431c6b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.708216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.709162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.709585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.709988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.710809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.713082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.720999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5d765858-b913-404e-9ebc-c0c2431c6b72 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:28.722611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5d765858-b913-404e-9ebc-c0c2431c6b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:28.736064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 978/1924] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:28.743851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cdc64-e73e-4e09-bde9-f9b06f0f3631 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.744138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cdc64-e73e-4e09-bde9-f9b06f0f3631 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.744448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cdc64-e73e-4e09-bde9-f9b06f0f3631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.747635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 947/1925] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:28 2022] POST /identity/v3/users => generated 299 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:28.749879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.753685 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:28.769305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8c0cdc64-e73e-4e09-bde9-f9b06f0f3631 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:28.769965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bf101-d391-4614-8325-82295a97fe27 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.770555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8c0cdc64-e73e-4e09-bde9-f9b06f0f3631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:28.776828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.778405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.779746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.780111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.780946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 979/1926] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:28.781068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.783759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.784220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cdc64-e73e-4e09-bde9-f9b06f0f3631 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.793751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cdc64-e73e-4e09-bde9-f9b06f0f3631 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.801124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 948/1927] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:28.802550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.805161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.805889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.806173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.806447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.807179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:28.807482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.809920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.824543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc718b13-dac0-40b7-a864-c75d5a3c0c7c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.831728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.832877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 980/1928] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:59:28.836005 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.836422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.836705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] PATH_INFO: `/v3/projects/65947b4c83b84514be2978453090cfa2/users/c2d4eb488aeb44928c1b06e754229e1e/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.838370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.861966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754716a5-c5be-4597-8e8a-99ebeb408d4b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:28.868198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65947b4c83b84514be2978453090cfa2, user_id=c2d4eb488aeb44928c1b06e754229e1e, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.870880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.875747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:28.878268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:28.878608 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:28.878838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:28.880794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:28.883331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:28.884465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:28.896716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 949/1929] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:28 2022] PUT /identity/v3/projects/65947b4c83b84514be2978453090cfa2/users/c2d4eb488aeb44928c1b06e754229e1e/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:28.897984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.920491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:28.932405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 981/1930] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:28 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:28.940519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:28.980617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b0be-d45d-4b2b-8b8e-f8a351748b84 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.028394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43310727-e844-4967-b3a3-8d421efe2da3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.034417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.035104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.035525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.035859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.037308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.039309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.070240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.071471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.071768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.072100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.073045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.074123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 982/1931] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:28 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:29.077263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.078057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.097972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 950/1932] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:29.098989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.100623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa14913-2a8e-48b9-bb9b-e5fbc1367f8b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.109672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2729a-4788-4937-abc3-d08272ff83f5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.111727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.112633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.112927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.113429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.114401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.115786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f483-6772-4358-936f-942065376b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.116618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.116837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f483-6772-4358-936f-942065376b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.117193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f483-6772-4358-936f-942065376b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.117550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f483-6772-4358-936f-942065376b46 admin admin] PATH_INFO: `/v3/projects/65947b4c83b84514be2978453090cfa2/users/c2d4eb488aeb44928c1b06e754229e1e/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.121020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f483-6772-4358-936f-942065376b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65947b4c83b84514be2978453090cfa2, user_id=c2d4eb488aeb44928c1b06e754229e1e, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.123736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f483-6772-4358-936f-942065376b46 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.140123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 983/1933] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:29.148609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.153983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 951/1934] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:29 2022] PUT /identity/v3/projects/65947b4c83b84514be2978453090cfa2/users/c2d4eb488aeb44928c1b06e754229e1e/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:29.163035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52638c0-7fd8-4ed4-81e4-c55a1042d62e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.165198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.165811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.167007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.206549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] MFA Rules not processed for user `c2d4eb488aeb44928c1b06e754229e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:29.207983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:29.241384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.241936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.242294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.242574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.243304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.245097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.262172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 984/1935] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:29.267943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.280808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5bb91-11ab-44e6-89b3-65388dd8eb9d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.287396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eefbd8-939c-4c51-acce-a26167c4c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.288288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eefbd8-939c-4c51-acce-a26167c4c672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.288654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eefbd8-939c-4c51-acce-a26167c4c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.288935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eefbd8-939c-4c51-acce-a26167c4c672 admin admin] PATH_INFO: `/v3/projects/9678dbba8a1c4f61875bd311334644f2/users/d1a796ec26e849f4b4e6027d7f667910/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.304422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 952/1936] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:29.307770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eefbd8-939c-4c51-acce-a26167c4c672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9678dbba8a1c4f61875bd311334644f2, user_id=d1a796ec26e849f4b4e6027d7f667910, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.310914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eefbd8-939c-4c51-acce-a26167c4c672 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.317922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.320134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:29.328559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 985/1937] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:29 2022] PUT /identity/v3/projects/9678dbba8a1c4f61875bd311334644f2/users/d1a796ec26e849f4b4e6027d7f667910/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:29.346582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745ac9f-7020-465b-9f99-f8c7ef9dd929 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.349881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745ac9f-7020-465b-9f99-f8c7ef9dd929 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.350829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745ac9f-7020-465b-9f99-f8c7ef9dd929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.356283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:29.397794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2745ac9f-7020-465b-9f99-f8c7ef9dd929 None None] MFA Rules not processed for user `d1a796ec26e849f4b4e6027d7f667910`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:29.399036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2745ac9f-7020-465b-9f99-f8c7ef9dd929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:29.427372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a5cd1-5c3b-4511-8aa7-5591b1157113 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.483523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:29.505323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.505996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.506282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.506561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.508509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:29.515254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 986/1938] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:29.521160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745ac9f-7020-465b-9f99-f8c7ef9dd929 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.577565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.579205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.589534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 953/1939] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:29.595610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745ac9f-7020-465b-9f99-f8c7ef9dd929 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.601092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.601847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.602135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.602415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.603120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.605348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.637791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 987/1940] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:29 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:29.642932 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.654181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dc0f7-586e-4ad1-a98e-83a276089c6f service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.732041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.732775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.733061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.733340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.734251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.737305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.762677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 954/1941] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:29 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:29.766217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.779960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b864d-9900-4710-8049-4dad35d16367 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.786169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.786799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.787216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.787945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.788227 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.790297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.809500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 988/1942] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:29.810966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.824976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8780ae-5e46-4689-97b5-2fae79562a35 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.832115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77be38-ad7e-4429-91c8-95ff37f15c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.833985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77be38-ad7e-4429-91c8-95ff37f15c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.834284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77be38-ad7e-4429-91c8-95ff37f15c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.834670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77be38-ad7e-4429-91c8-95ff37f15c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.835833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77be38-ad7e-4429-91c8-95ff37f15c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.837821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77be38-ad7e-4429-91c8-95ff37f15c86 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.854907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 955/1943] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:29.858816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:29.871068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9820a2-0761-485c-9c10-2067247c1bb1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:29.875926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:29.877464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.877886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.878244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] PATH_INFO: `/v3/projects/27122bc95b7c4316970270136908a436/users/8718d54823294972b15b0570a9986af2/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.895274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27122bc95b7c4316970270136908a436, user_id=8718d54823294972b15b0570a9986af2, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:29.898040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:29.913043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 989/1944] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:29 2022] PUT /identity/v3/projects/27122bc95b7c4316970270136908a436/users/8718d54823294972b15b0570a9986af2/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:29.924508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfbdb-4d49-49be-8fd7-b65dea3aaea7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:29.924871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfbdb-4d49-49be-8fd7-b65dea3aaea7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:29.925299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cfbdb-4d49-49be-8fd7-b65dea3aaea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:29.933134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:29.957878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f78cfbdb-4d49-49be-8fd7-b65dea3aaea7 None None] MFA Rules not processed for user `8718d54823294972b15b0570a9986af2`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:29.959250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f78cfbdb-4d49-49be-8fd7-b65dea3aaea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:30.055966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 956/1945] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:30.058077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:30.128841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123a61-cd9c-449f-91fb-a7a8642fd7f5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:30.133615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:30.134282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.134562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.134841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.135893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:30.137807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:30.166048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 990/1946] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:30 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:30.170093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cfbdb-4d49-49be-8fd7-b65dea3aaea7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:30.181176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cfbdb-4d49-49be-8fd7-b65dea3aaea7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:30.253304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:30.253848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.254187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.254486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.255098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:30.257163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:30.281246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 957/1947] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:30 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:30.285260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:30.296056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41df7a5-f0d9-42b9-9d85-e623a2ab4fef admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:30.301530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:30.302194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.302477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.302751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.303513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:30.305149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:30.316947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 991/1948] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:30.320858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:30.331113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77adf-e30a-4f3b-95d2-8959089b68ca admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:30.335923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:30.336894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.337178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.337458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] PATH_INFO: `/v3/projects/112163d87ab9416481ca3fcc2e5baa3f/users/1bb3a073f62e44b585a1f9949d43aaf5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.355342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=112163d87ab9416481ca3fcc2e5baa3f, user_id=1bb3a073f62e44b585a1f9949d43aaf5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:30.358095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:30.372123 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 958/1949] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:30 2022] PUT /identity/v3/projects/112163d87ab9416481ca3fcc2e5baa3f/users/1bb3a073f62e44b585a1f9949d43aaf5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:30.376875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:30.383656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:30.398257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839238b6-8d30-4194-a26c-91d365d594a4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:30.461270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f8f83-7d00-46a5-8492-5d5efa88bd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:30.461868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f8f83-7d00-46a5-8492-5d5efa88bd4c admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.462154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f8f83-7d00-46a5-8492-5d5efa88bd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.462433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f8f83-7d00-46a5-8492-5d5efa88bd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.463167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f8f83-7d00-46a5-8492-5d5efa88bd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:30.464804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f8f83-7d00-46a5-8492-5d5efa88bd4c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:30.476547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 992/1950] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:30.480503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:30.492574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f503c-b8e5-4e1b-9ed8-3e0f1ea600a7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:30.497577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:30.498476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.498757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.499031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] PATH_INFO: `/v3/projects/112163d87ab9416481ca3fcc2e5baa3f/users/1bb3a073f62e44b585a1f9949d43aaf5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.501387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] RBAC: Authorizing `identity:create_grant(project_id=112163d87ab9416481ca3fcc2e5baa3f, user_id=1bb3a073f62e44b585a1f9949d43aaf5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:30.504176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:30.518645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 959/1951] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:30 2022] PUT /identity/v3/projects/112163d87ab9416481ca3fcc2e5baa3f/users/1bb3a073f62e44b585a1f9949d43aaf5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:30.529771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dba9a-7553-4fba-939b-d5631e5089f5 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.530058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dba9a-7553-4fba-939b-d5631e5089f5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.530336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dba9a-7553-4fba-939b-d5631e5089f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.558854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b36dba9a-7553-4fba-939b-d5631e5089f5 None None] MFA Rules not processed for user `1bb3a073f62e44b585a1f9949d43aaf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:30.560127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b36dba9a-7553-4fba-939b-d5631e5089f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:30.629085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 993/1952] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:30.639678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:30.690904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:30.697537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f6c1-7f2b-4260-a3d8-15201f6faade admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:30.701866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:30.702798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:30.703177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:30.703502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:30.717391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:30.718756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:30.729563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 960/1953] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:31.309291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dba9a-7553-4fba-939b-d5631e5089f5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:31.318690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dba9a-7553-4fba-939b-d5631e5089f5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:31.322247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe3084-30cd-4876-b76b-685bc4d60858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:31.322883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3084-30cd-4876-b76b-685bc4d60858 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:31.323164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3084-30cd-4876-b76b-685bc4d60858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:31.323438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3084-30cd-4876-b76b-685bc4d60858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:31.389801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe3084-30cd-4876-b76b-685bc4d60858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:31.391224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe3084-30cd-4876-b76b-685bc4d60858 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:31.400420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 994/1954] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:33.882783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:33.884405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:33.974228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01bec9-5743-48b1-84b4-b38680b655a9 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:33.980157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:33.981096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:33.981395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:33.981704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] PATH_INFO: `/v3/users/74408245319a4075b79d263326e19c84` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:33.982929 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] RBAC: Authorizing `identity:delete_user(user_id=74408245319a4075b79d263326e19c84)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:33.984883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:34.062936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74408245319a4075b79d263326e19c84'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.064705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74408245319a4075b79d263326e19c84'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.073614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74408245319a4075b79d263326e19c84'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.075054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74408245319a4075b79d263326e19c84'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.087618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74408245319a4075b79d263326e19c84'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.113440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 961/1955] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:33 2022] DELETE /identity/v3/users/74408245319a4075b79d263326e19c84 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:34.119007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:34.119747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:34.120061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:34.175711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:34.177451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:34.213346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:34.285852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 995/1956] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:34.297134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:34.299616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:34.434956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184a2a4-ced6-4c01-9b43-37a90f6d206e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:34.441698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:34.442964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:34.443411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:34.443695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:34.458637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:34.460126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:34.472460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 962/1957] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:34.787990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:34.789642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:34.808442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197607eb-5c46-4e15-8d3c-90946c2263ed None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:34.815809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:34.817442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:34.817820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:34.818193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] PATH_INFO: `/v3/projects/a16c728ace3f4df5a25c28c3688dac6c` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:34.819521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16c728ace3f4df5a25c28c3688dac6c)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:34.822263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:34.888084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16c728ace3f4df5a25c28c3688dac6c', 'request_id': 'req-b146232a-9378-4913-90af-b693b1e6ef2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.896020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16c728ace3f4df5a25c28c3688dac6c', 'request_id': 'req-b146232a-9378-4913-90af-b693b1e6ef2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.903437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16c728ace3f4df5a25c28c3688dac6c', 'request_id': 'req-b146232a-9378-4913-90af-b693b1e6ef2c'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.918403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] The token cache is being invalidate because project a16c728ace3f4df5a25c28c3688dac6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:34.919341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.919714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:34.922334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 996/1958] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:34 2022] DELETE /identity/v3/projects/a16c728ace3f4df5a25c28c3688dac6c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:34.927729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:34.947287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d77f3-1a4b-474e-a7dd-f48b523f688e service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:34.960814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:35.031983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:35.033418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:35.033783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:35.034064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] PATH_INFO: `/v3/users/e226cf04a03b4743b51d88a9e3b94515` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:35.035305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e226cf04a03b4743b51d88a9e3b94515)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:35.037254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:35.088828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e226cf04a03b4743b51d88a9e3b94515'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.089491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e226cf04a03b4743b51d88a9e3b94515'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.097174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e226cf04a03b4743b51d88a9e3b94515'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.097814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e226cf04a03b4743b51d88a9e3b94515'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.105977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e226cf04a03b4743b51d88a9e3b94515'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.124303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 963/1959] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:34 2022] DELETE /identity/v3/users/e226cf04a03b4743b51d88a9e3b94515 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:35.600788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:35.603002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:35.622648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146232a-9378-4913-90af-b693b1e6ef2c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:35.689666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:35.691649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:35.727032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:35.727032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:35.727032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:35.727032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] PATH_INFO: `/v3/projects/b172db494d7d4fff89cc83f56c41eeac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:35.727329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b172db494d7d4fff89cc83f56c41eeac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:35.727410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:35.769374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ed66-a169-4f0d-875c-88ff090fcc0e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:35.782691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b172db494d7d4fff89cc83f56c41eeac', 'request_id': 'req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.784064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:35.790549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b172db494d7d4fff89cc83f56c41eeac', 'request_id': 'req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.803304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b172db494d7d4fff89cc83f56c41eeac', 'request_id': 'req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.817129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] The token cache is being invalidate because project b172db494d7d4fff89cc83f56c41eeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:35.817463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.818093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ff3c19e6-d6c4-443d-bcb0-51cf59f5a8c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:35.819803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 997/1960] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:35 2022] DELETE /identity/v3/projects/b172db494d7d4fff89cc83f56c41eeac => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:35.826000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:35.826294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:35.826787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:35.830640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:35.844118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a0893-d211-4899-b3ce-0c85220d5bbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:35.844976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a0893-d211-4899-b3ce-0c85220d5bbc service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:35.845313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a0893-d211-4899-b3ce-0c85220d5bbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:35.845593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a0893-d211-4899-b3ce-0c85220d5bbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:35.847440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a07a0893-d211-4899-b3ce-0c85220d5bbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:35.850509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:35.851681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:35.921632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 998/1961] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:35.922944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:35.925077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:35.925986 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a0893-d211-4899-b3ce-0c85220d5bbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:35.927721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a0893-d211-4899-b3ce-0c85220d5bbc service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:35.938200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389739ee-00f7-434e-b02a-79f4e9f5c019 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:35.940789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 964/1962] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:35.944477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:35.945361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:35.945639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:35.945915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] PATH_INFO: `/v3/users/25a9500348074250aea0a517f8139938` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:35.947194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=25a9500348074250aea0a517f8139938)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:35.949142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:35.950923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec51b-0b3e-4037-8ecc-a281855c4afb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:35.951383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec51b-0b3e-4037-8ecc-a281855c4afb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:35.952049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec51b-0b3e-4037-8ecc-a281855c4afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:35.980230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-70eec51b-0b3e-4037-8ecc-a281855c4afb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:35.981446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-70eec51b-0b3e-4037-8ecc-a281855c4afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:36.003539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 965/1963] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:36.007426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec51b-0b3e-4037-8ecc-a281855c4afb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.046491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25a9500348074250aea0a517f8139938'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:36.048260 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25a9500348074250aea0a517f8139938'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:36.048525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec51b-0b3e-4037-8ecc-a281855c4afb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.050522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25a9500348074250aea0a517f8139938'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:36.064386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.067165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.067165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.067165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.067507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:36.067815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.069461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.076237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25a9500348074250aea0a517f8139938'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:36.112823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5c54b963-e84a-46de-944c-53c6453b0cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25a9500348074250aea0a517f8139938'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:36.117999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 966/1964] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:36.129356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.133338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:36.152876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 999/1965] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:35 2022] DELETE /identity/v3/users/25a9500348074250aea0a517f8139938 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:36.158118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.159910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.159910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.169223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f21ee-1844-4e84-9a57-2db38f6b6bae admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.192429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:36.194216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:36.272444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:36.274717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.275446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.275718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.275998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.276887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.279418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.297022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1000/1966] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:36.298538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.299884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:36.316826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 967/1967] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:36 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:36.317966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.321501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:36.372179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f2c3f-ed1c-448c-b2ab-4dc585abadf2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.404607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509194f-c460-450d-9568-456173c0dd26 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.413423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.429782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.430097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.430464 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.445983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.447347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.447851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.448320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.451445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:36.474979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:36.546023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.546744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.549146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.549969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.562273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1001/1968] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:36.562985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.563749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 968/1969] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:36.576171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ba094-2931-4acd-8c1b-08cc461d091c service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.582189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.582976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.583344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.583625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.584325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.586457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.624592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1002/1970] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:36 2022] POST /identity/v3/users => generated 294 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:36.626500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.639127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603ccb3-39af-4e33-a504-fea00e798210 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.659482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.660179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.660496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.660777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.661511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.664415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.700906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 969/1971] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:36.706793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.724833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603678dc-044e-4736-9121-c061aeaef3fe admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.740110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.741534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.744255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.744834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.745809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.750039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.818680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1003/1972] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:36.820133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.837482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9354148-0938-4518-8ad2-1bebb526720e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.853655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce786a8-988f-4f56-8984-111659253368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:36.855369 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce786a8-988f-4f56-8984-111659253368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.855669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce786a8-988f-4f56-8984-111659253368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.855945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce786a8-988f-4f56-8984-111659253368 admin admin] PATH_INFO: `/v3/projects/1a4fa86d630d471e840f69625114fe77/users/1b7f9d8bc8be496cbda647c8d15f40b3/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:36.902879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce786a8-988f-4f56-8984-111659253368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4fa86d630d471e840f69625114fe77, user_id=1b7f9d8bc8be496cbda647c8d15f40b3, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:36.912063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce786a8-988f-4f56-8984-111659253368 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:36.913370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:36.915092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:36.957200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94272e53-f099-45ca-99e3-e8f5a5880579 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:36.961700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 970/1973] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:36 2022] PUT /identity/v3/projects/1a4fa86d630d471e840f69625114fe77/users/1b7f9d8bc8be496cbda647c8d15f40b3/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:36.973356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b15bbb-02d7-48ab-b13a-d5f8450d23c2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:36.973653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b15bbb-02d7-48ab-b13a-d5f8450d23c2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:36.973920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b15bbb-02d7-48ab-b13a-d5f8450d23c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.034694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-19b15bbb-02d7-48ab-b13a-d5f8450d23c2 None None] MFA Rules not processed for user `1b7f9d8bc8be496cbda647c8d15f40b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:37.036580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:37.038509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-19b15bbb-02d7-48ab-b13a-d5f8450d23c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:37.130635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:37.132677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:37.132974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:37.133254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] PATH_INFO: `/v3/projects/5233e79acf774ccda021282775c90e64` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.138317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5233e79acf774ccda021282775c90e64)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:37.140067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:37.216950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 971/1974] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:37.222936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b15bbb-02d7-48ab-b13a-d5f8450d23c2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:37.240084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b15bbb-02d7-48ab-b13a-d5f8450d23c2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:37.264002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:37.264954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:37.267316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:37.267870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.269178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:37.275094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:37.317249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5233e79acf774ccda021282775c90e64', 'request_id': 'req-6b6f0044-5676-4be8-83dd-907278ea57e4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.331317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5233e79acf774ccda021282775c90e64', 'request_id': 'req-6b6f0044-5676-4be8-83dd-907278ea57e4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.339628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 972/1975] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:37 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:37.344075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:37.346638 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5233e79acf774ccda021282775c90e64', 'request_id': 'req-6b6f0044-5676-4be8-83dd-907278ea57e4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.377279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] The token cache is being invalidate because project 5233e79acf774ccda021282775c90e64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:37.377596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.380169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.383170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1004/1976] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:36 2022] DELETE /identity/v3/projects/5233e79acf774ccda021282775c90e64 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:37.386730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:37.389358 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:37.439455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:37.471916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ccc99-5c43-4360-ae18-e3c0f39f6f75 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:37.478027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:37.478851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:37.479616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:37.479987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.480904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:37.483193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:37.506939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f0044-5676-4be8-83dd-907278ea57e4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:37.511983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:37.512947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:37.513178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:37.513456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] PATH_INFO: `/v3/users/c94dfcbfcb114d4caef320b88586a407` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.514053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 973/1977] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:37 2022] POST /identity/v3/users => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:37.514964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94dfcbfcb114d4caef320b88586a407)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:37.517759 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:37.520138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:37.523976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:37.549450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f40bbc-9825-4531-af86-caea548ea6cc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:37.566587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:37.566587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:37.566587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:37.566587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.567120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:37.567120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:37.581319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 974/1978] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:37.582270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:37.595981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31f92-468a-47fa-a632-9644aa635ca5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:37.602534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:37.604130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:37.605108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:37.605732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] PATH_INFO: `/v3/projects/9c444f4180184a0d849e83eb25b50f8b/users/8f708b7ceb28451fabe845129c97e2a5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.608533 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94dfcbfcb114d4caef320b88586a407'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.609243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94dfcbfcb114d4caef320b88586a407'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.609876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94dfcbfcb114d4caef320b88586a407'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.622304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94dfcbfcb114d4caef320b88586a407'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.629316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c444f4180184a0d849e83eb25b50f8b, user_id=8f708b7ceb28451fabe845129c97e2a5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:37.632403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:37.634755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:37.643705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94dfcbfcb114d4caef320b88586a407'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:37.664340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 975/1979] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:37 2022] PUT /identity/v3/projects/9c444f4180184a0d849e83eb25b50f8b/users/8f708b7ceb28451fabe845129c97e2a5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:37.668919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1005/1980] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:37 2022] DELETE /identity/v3/users/c94dfcbfcb114d4caef320b88586a407 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:37.676067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:37.680836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:37.857880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632aa96-5058-4f83-9963-7377816d2c9d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:37.947988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5985-7463-4f9d-b967-a84e7828a6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:37.948776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5985-7463-4f9d-b967-a84e7828a6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:37.949059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5985-7463-4f9d-b967-a84e7828a6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:37.949551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5985-7463-4f9d-b967-a84e7828a6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:37.950292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5985-7463-4f9d-b967-a84e7828a6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:37.952425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5985-7463-4f9d-b967-a84e7828a6f2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:37.967900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 976/1981] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:37.972758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:38.013202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d549580-9b6f-42f2-a2ab-51e20b9fdb0d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:38.026084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:38.028667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.028919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.029214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] PATH_INFO: `/v3/projects/9c444f4180184a0d849e83eb25b50f8b/users/8f708b7ceb28451fabe845129c97e2a5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.043325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c444f4180184a0d849e83eb25b50f8b, user_id=8f708b7ceb28451fabe845129c97e2a5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:38.046310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:38.078899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1006/1982] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:37 2022] PUT /identity/v3/projects/9c444f4180184a0d849e83eb25b50f8b/users/8f708b7ceb28451fabe845129c97e2a5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:38.079959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.079959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.079959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.118029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] MFA Rules not processed for user `8f708b7ceb28451fabe845129c97e2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:38.120188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:38.182456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:38.231340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:38.231597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:38.250715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 977/1983] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:38.260060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:38.261689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:38.342874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f490ec7-b41d-4527-a9ef-eeac06232438 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:38.348392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:38.349273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.349992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.349992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] PATH_INFO: `/v3/projects/6d42a3e16dcd412d8058abe58ac159c4` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.351067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d42a3e16dcd412d8058abe58ac159c4)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:38.352953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:38.387943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac27d6-3ea8-467a-9974-e3de63c34933 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:38.392186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e675e6e-d3fe-4f3a-8f6b-fa3a042266eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:38.393213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675e6e-d3fe-4f3a-8f6b-fa3a042266eb service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.393499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675e6e-d3fe-4f3a-8f6b-fa3a042266eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.393772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675e6e-d3fe-4f3a-8f6b-fa3a042266eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.416778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d42a3e16dcd412d8058abe58ac159c4', 'request_id': 'req-0bb63d73-0272-44c5-811c-f36006405089'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.417699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e675e6e-d3fe-4f3a-8f6b-fa3a042266eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:38.419480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e675e6e-d3fe-4f3a-8f6b-fa3a042266eb service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:38.429289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d42a3e16dcd412d8058abe58ac159c4', 'request_id': 'req-0bb63d73-0272-44c5-811c-f36006405089'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.441256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d42a3e16dcd412d8058abe58ac159c4', 'request_id': 'req-0bb63d73-0272-44c5-811c-f36006405089'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.443948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 978/1984] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:38.472819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] The token cache is being invalidate because project 6d42a3e16dcd412d8058abe58ac159c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:38.472819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.472819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-0bb63d73-0272-44c5-811c-f36006405089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.474659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1007/1985] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:38 2022] DELETE /identity/v3/projects/6d42a3e16dcd412d8058abe58ac159c4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:38.483953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0f40-f681-4864-a633-1e29a963ab6a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.483953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0f40-f681-4864-a633-1e29a963ab6a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.483953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0f40-f681-4864-a633-1e29a963ab6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.513392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d65d0f40-f681-4864-a633-1e29a963ab6a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:38.514802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d65d0f40-f681-4864-a633-1e29a963ab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:38.603036 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 979/1986] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:38.612972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d897de5-d8cf-45fc-9a47-3e0ba6fe57ae None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.613354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d897de5-d8cf-45fc-9a47-3e0ba6fe57ae None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.613686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d897de5-d8cf-45fc-9a47-3e0ba6fe57ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.652193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3d897de5-d8cf-45fc-9a47-3e0ba6fe57ae None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:38.653800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3d897de5-d8cf-45fc-9a47-3e0ba6fe57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:38.670919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1008/1987] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:38.675453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d0f40-f681-4864-a633-1e29a963ab6a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:38.692404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d0f40-f681-4864-a633-1e29a963ab6a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:38.698544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:38.699219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.699693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.700034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.700814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:38.701120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:38.702734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:38.723362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 980/1988] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:38.728295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d897de5-d8cf-45fc-9a47-3e0ba6fe57ae None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:38.729482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:38.732558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:38.751613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c656eef-7253-47a1-b08a-254a14c842dc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:38.756429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d897de5-d8cf-45fc-9a47-3e0ba6fe57ae None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:38.760434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:38.762178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.762463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.762793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] PATH_INFO: `/v3/users/1b7f9d8bc8be496cbda647c8d15f40b3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.764806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7f9d8bc8be496cbda647c8d15f40b3)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:38.767859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:38.771185 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:38.772654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.773379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.773838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.774735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:38.778196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:38.820421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1009/1989] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:38 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:38.824933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:38.851258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7f9d8bc8be496cbda647c8d15f40b3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.852072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7f9d8bc8be496cbda647c8d15f40b3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.864597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7f9d8bc8be496cbda647c8d15f40b3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.865416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7f9d8bc8be496cbda647c8d15f40b3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.867547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:38.874612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ef7fc33a-09fd-470c-b967-2debe53a4b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7f9d8bc8be496cbda647c8d15f40b3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:38.887624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:38.919371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 981/1990] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:38 2022] DELETE /identity/v3/users/1b7f9d8bc8be496cbda647c8d15f40b3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:38.924636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:38.924959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:38.925234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:38.950692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7852a49-e7fd-441d-9256-be4ce4d95dc0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:38.959383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:38.961023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.050904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 982/1991] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:39.052026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.057379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.074753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.075492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.076880 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.077170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.077974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.080889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e9bc7-db6d-44ec-a06c-a67b8a325e74 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.081921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.086866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.088609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.089097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.089558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] PATH_INFO: `/v3/users/d1a796ec26e849f4b4e6027d7f667910` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.091122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a796ec26e849f4b4e6027d7f667910)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.093378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.115898 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1010/1992] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:38 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:39.121989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.128327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.190539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a796ec26e849f4b4e6027d7f667910'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.191616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a796ec26e849f4b4e6027d7f667910'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.209499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a796ec26e849f4b4e6027d7f667910'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.210530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a796ec26e849f4b4e6027d7f667910'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.223913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a796ec26e849f4b4e6027d7f667910'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.231044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef7cc4-f0a3-4e64-932b-5b70c8c4d64a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.235442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3e32d-a1ae-49bb-9764-b189321a214d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.236261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3e32d-a1ae-49bb-9764-b189321a214d service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.236602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3e32d-a1ae-49bb-9764-b189321a214d service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.237022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3e32d-a1ae-49bb-9764-b189321a214d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.239012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dcf3e32d-a1ae-49bb-9764-b189321a214d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.249126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 983/1993] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:39 2022] DELETE /identity/v3/users/d1a796ec26e849f4b4e6027d7f667910 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:39.252495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.253014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.255441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3e32d-a1ae-49bb-9764-b189321a214d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.257448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3e32d-a1ae-49bb-9764-b189321a214d service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.272504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1011/1994] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:39.274640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057470e-7564-4575-bffb-ec987b5a270b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.274923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057470e-7564-4575-bffb-ec987b5a270b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.275349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057470e-7564-4575-bffb-ec987b5a270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.301760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9057470e-7564-4575-bffb-ec987b5a270b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:39.304643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9057470e-7564-4575-bffb-ec987b5a270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.327009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:39.355294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf5cdb-fc6b-4b53-b9bb-640a665f6ad1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.362153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.362743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.363040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.363437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.364173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.365889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.379513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 984/1995] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:39.384644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.398002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1012/1996] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:39.401223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef271cda-a1a6-44ce-ba38-b6a3ed7a2121 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.410221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.411176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.411748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.412333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.413346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.415752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.432330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 985/1997] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:39.433662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057470e-7564-4575-bffb-ec987b5a270b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.439277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.442223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.457257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057470e-7564-4575-bffb-ec987b5a270b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.463977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f3268-edf3-4e3f-af08-49f7556c1561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.464578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f3268-edf3-4e3f-af08-49f7556c1561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.464868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f3268-edf3-4e3f-af08-49f7556c1561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.465135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f3268-edf3-4e3f-af08-49f7556c1561 admin admin] PATH_INFO: `/v3/projects/22adbf95e448494fadeacb9a0a3fdb4e/users/89187bc6f2984409b58670e2cba0d941/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.468284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:39.492428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f3268-edf3-4e3f-af08-49f7556c1561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22adbf95e448494fadeacb9a0a3fdb4e, user_id=89187bc6f2984409b58670e2cba0d941, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.496963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f3268-edf3-4e3f-af08-49f7556c1561 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.518978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1013/1998] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:39 2022] PUT /identity/v3/projects/22adbf95e448494fadeacb9a0a3fdb4e/users/89187bc6f2984409b58670e2cba0d941/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:39.528646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e2da5-e55f-47bb-9d80-9387ab05cc1a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.531872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.532136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.532645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.567719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] MFA Rules not processed for user `89187bc6f2984409b58670e2cba0d941`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:39.569061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.599915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.601437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.601796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.602147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.659702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1014/1999] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:39.676547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.684664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.697228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.700145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.735844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 986/2000] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:39.737476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.739804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:39.764938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b073c-d778-4988-a530-d6fa69cb0fca service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.774587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.775851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.775851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.776061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] PATH_INFO: `/v3/projects/1a4fa86d630d471e840f69625114fe77` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.777557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4fa86d630d471e840f69625114fe77)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.779480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.795399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ce5ef-7fab-495d-95ce-f2d0e90f0ae4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.800526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:39.801264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:39.801773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:39.802119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:39.815175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:39.817379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:39.847337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1015/2001] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:39.861701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.862295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:39.885807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4fa86d630d471e840f69625114fe77', 'request_id': 'req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.886145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471539b8-a7a0-4fe5-a9c9-fc5b981743b4 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:39.898012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4fa86d630d471e840f69625114fe77', 'request_id': 'req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.917843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4fa86d630d471e840f69625114fe77', 'request_id': 'req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.935572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] The token cache is being invalidate because project 1a4fa86d630d471e840f69625114fe77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:39.936086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.937155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:39.940270 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 987/2002] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:39 2022] DELETE /identity/v3/projects/1a4fa86d630d471e840f69625114fe77 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:39.945081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:39.947614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:40.018518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:40.019469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:40.019752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:40.020347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] PATH_INFO: `/v3/users/89187bc6f2984409b58670e2cba0d941` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:40.021429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] RBAC: Authorizing `identity:delete_user(user_id=89187bc6f2984409b58670e2cba0d941)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:40.025096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:40.047022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:40.075815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf49d5-f6ac-49ee-b32a-a782d87c26e2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:40.083386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:40.103860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:40.104146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:40.104892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] PATH_INFO: `/v3/users/8f708b7ceb28451fabe845129c97e2a5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:40.109234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f708b7ceb28451fabe845129c97e2a5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:40.114930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:40.187331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89187bc6f2984409b58670e2cba0d941'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.189928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89187bc6f2984409b58670e2cba0d941'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.191028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89187bc6f2984409b58670e2cba0d941'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.202752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89187bc6f2984409b58670e2cba0d941'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.217002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f708b7ceb28451fabe845129c97e2a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.217773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f708b7ceb28451fabe845129c97e2a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.222808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89187bc6f2984409b58670e2cba0d941'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.228163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f708b7ceb28451fabe845129c97e2a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.229076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f708b7ceb28451fabe845129c97e2a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.240303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-13d7edb3-9742-40c3-835e-5adc380f529e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f708b7ceb28451fabe845129c97e2a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.240588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1016/2003] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:39 2022] DELETE /identity/v3/users/89187bc6f2984409b58670e2cba0d941 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:40.241605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:40.243074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:40.265994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 988/2004] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:39 2022] DELETE /identity/v3/users/8f708b7ceb28451fabe845129c97e2a5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:40.268316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:40.268811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:40.269306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:40.289411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:40.291483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:40.333637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d74f86-7896-4e64-a220-ccf05681987a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:40.334305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:40.384253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 989/2005] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:40.397937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:40.400407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:40.436404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:40.437468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:40.437819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:40.438836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] PATH_INFO: `/v3/projects/9678dbba8a1c4f61875bd311334644f2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:40.440903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9678dbba8a1c4f61875bd311334644f2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:40.442938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:40.505477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:40.513035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9678dbba8a1c4f61875bd311334644f2', 'request_id': 'req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.523056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9678dbba8a1c4f61875bd311334644f2', 'request_id': 'req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.534945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9678dbba8a1c4f61875bd311334644f2', 'request_id': 'req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.561461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] The token cache is being invalidate because project 9678dbba8a1c4f61875bd311334644f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:40.561930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.562922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.566946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1017/2006] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:40 2022] DELETE /identity/v3/projects/9678dbba8a1c4f61875bd311334644f2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:40.571460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:40.574881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:40.593783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e236a1-7473-425b-8f14-b68da9faddf6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:40.679052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80f32-5d9e-4ad3-9de5-c21f321067d5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:40.685392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:40.686561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:40.687063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:40.687542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] PATH_INFO: `/v3/users/8718d54823294972b15b0570a9986af2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:40.689031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8718d54823294972b15b0570a9986af2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:40.692158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:40.711773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:40.713180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:40.715173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:40.715461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:40.720953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:40.742607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:40.745099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:40.756572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8718d54823294972b15b0570a9986af2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.757362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8718d54823294972b15b0570a9986af2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.758358 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8718d54823294972b15b0570a9986af2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.772003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8718d54823294972b15b0570a9986af2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.773150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 990/2007] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:40.794308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9864f390-1b91-4561-a281-c041c9f30065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8718d54823294972b15b0570a9986af2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:40.798664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:40.800424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:40.812101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1018/2008] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:40 2022] DELETE /identity/v3/users/8718d54823294972b15b0570a9986af2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:40.821975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094132cc-36e1-444d-87f7-99aa74ac505e service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:40.906472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:40.906891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:40.907813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:40.908119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] PATH_INFO: `/v3/projects/9c444f4180184a0d849e83eb25b50f8b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:40.909870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c444f4180184a0d849e83eb25b50f8b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:40.912626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:40.983147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:41.016716 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c444f4180184a0d849e83eb25b50f8b', 'request_id': 'req-abb88154-22b7-4eaa-b764-dfd38e2b81f9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.038003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c444f4180184a0d849e83eb25b50f8b', 'request_id': 'req-abb88154-22b7-4eaa-b764-dfd38e2b81f9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.054853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c444f4180184a0d849e83eb25b50f8b', 'request_id': 'req-abb88154-22b7-4eaa-b764-dfd38e2b81f9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.073388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] The token cache is being invalidate because project 9c444f4180184a0d849e83eb25b50f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:41.079908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.081974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-abb88154-22b7-4eaa-b764-dfd38e2b81f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.084782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 991/2009] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:40 2022] DELETE /identity/v3/projects/9c444f4180184a0d849e83eb25b50f8b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:41.090986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59039-4f02-4eda-b8c6-d4c4633ebc7f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.092276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59039-4f02-4eda-b8c6-d4c4633ebc7f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.095684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59039-4f02-4eda-b8c6-d4c4633ebc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.123719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8ff59039-4f02-4eda-b8c6-d4c4633ebc7f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:41.125558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8ff59039-4f02-4eda-b8c6-d4c4633ebc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.257400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:41.307165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1019/2010] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:41.309448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69185-0352-454c-acfc-45161e5f859f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.310027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69185-0352-454c-acfc-45161e5f859f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.310835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69185-0352-454c-acfc-45161e5f859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.332741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-74a69185-0352-454c-acfc-45161e5f859f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:41.334127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-74a69185-0352-454c-acfc-45161e5f859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.344650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 992/2011] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:41.349921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff59039-4f02-4eda-b8c6-d4c4633ebc7f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:41.365707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff59039-4f02-4eda-b8c6-d4c4633ebc7f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:41.372966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:41.373958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.374263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.374595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.375348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:41.377043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:41.379422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:41.403733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1020/2012] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:41.404164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69185-0352-454c-acfc-45161e5f859f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:41.440675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69185-0352-454c-acfc-45161e5f859f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:41.450390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:41.458007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.458368 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.458644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.461062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:41.461639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:41.463919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.468305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:41.490818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034f286-8462-4004-a8d2-f2d4def5aa6e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:41.496777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:41.497748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.498094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.498595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] PATH_INFO: `/v3/projects/27122bc95b7c4316970270136908a436` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.501324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27122bc95b7c4316970270136908a436)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:41.503373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:41.528397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 993/2013] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:41 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:41.529330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:41.542028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.585583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27122bc95b7c4316970270136908a436', 'request_id': 'req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.591807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27122bc95b7c4316970270136908a436', 'request_id': 'req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.599368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27122bc95b7c4316970270136908a436', 'request_id': 'req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.615442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] The token cache is being invalidate because project 27122bc95b7c4316970270136908a436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:41.615735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.616692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.618765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1021/2014] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:41 2022] DELETE /identity/v3/projects/27122bc95b7c4316970270136908a436 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:41.619515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:41.620895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.636179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65380a1-3427-4411-98d8-d74a5c1cbb04 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:41.697258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff133c1-adee-4348-9e61-51bf2b2ac5d8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:41.709919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:41.710689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.711035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.711372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.712279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:41.714618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:41.724766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:41.726282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.726651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.727125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] PATH_INFO: `/v3/projects/22adbf95e448494fadeacb9a0a3fdb4e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.728507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22adbf95e448494fadeacb9a0a3fdb4e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:41.731351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:41.743751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1022/2015] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:41 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:41.745047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:41.748223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.764150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabda13d-e48e-435f-86f5-244eb4f53cb5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:41.772164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:41.773525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.773953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.774377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] PATH_INFO: `/v3/users/1bb3a073f62e44b585a1f9949d43aaf5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.776269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb3a073f62e44b585a1f9949d43aaf5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:41.778445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:41.804517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22adbf95e448494fadeacb9a0a3fdb4e', 'request_id': 'req-1efed70d-814a-4938-b60d-b82619d788d8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.805604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:41.814988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22adbf95e448494fadeacb9a0a3fdb4e', 'request_id': 'req-1efed70d-814a-4938-b60d-b82619d788d8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.825658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22adbf95e448494fadeacb9a0a3fdb4e', 'request_id': 'req-1efed70d-814a-4938-b60d-b82619d788d8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.842024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] The token cache is being invalidate because project 22adbf95e448494fadeacb9a0a3fdb4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:41.842321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.843353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.844702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:41.850113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 994/2016] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:41 2022] DELETE /identity/v3/projects/22adbf95e448494fadeacb9a0a3fdb4e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:41.850113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:41.851123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb3a073f62e44b585a1f9949d43aaf5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.851238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.851972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb3a073f62e44b585a1f9949d43aaf5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.852884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb3a073f62e44b585a1f9949d43aaf5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.868469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb3a073f62e44b585a1f9949d43aaf5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.880697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d24b207e-6fef-46cd-9a4f-92f6f1116584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb3a073f62e44b585a1f9949d43aaf5'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:41.893472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1023/2017] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:41 2022] DELETE /identity/v3/users/1bb3a073f62e44b585a1f9949d43aaf5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:41.894740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d1233-9120-40f5-b437-77c237ddde7b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:41.895030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d1233-9120-40f5-b437-77c237ddde7b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:41.895297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d1233-9120-40f5-b437-77c237ddde7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:41.914780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-595d1233-9120-40f5-b437-77c237ddde7b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:41.916025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-595d1233-9120-40f5-b437-77c237ddde7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:41.924418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efed70d-814a-4938-b60d-b82619d788d8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.013790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.014662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.015218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.015626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1024/2018] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:42.015907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.017377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.019438 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.024643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5f9a7-8e96-4267-bd89-8a83c5f81747 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.025005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5f9a7-8e96-4267-bd89-8a83c5f81747 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.025289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5f9a7-8e96-4267-bd89-8a83c5f81747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.031424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 995/2019] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:42.035646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.037484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.052068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a8e5f9a7-8e96-4267-bd89-8a83c5f81747 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:42.052860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442bf86-6af5-44a3-a442-2ff16275eb7c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.053508 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a8e5f9a7-8e96-4267-bd89-8a83c5f81747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.067189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1025/2020] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:42.068147 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5f9a7-8e96-4267-bd89-8a83c5f81747 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.070709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.071511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.072336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.072336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.073130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.074927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.077524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5f9a7-8e96-4267-bd89-8a83c5f81747 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.083473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.084387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.084504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.084687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.085406 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:42.085704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.087309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.091851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 996/2021] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:42.095697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.106694 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1026/2022] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:42.107883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.109709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e24168-b680-4063-b941-c392fa385a58 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.115347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd0c7c-9a26-4833-9587-55a139f413ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.116420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd0c7c-9a26-4833-9587-55a139f413ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.116711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd0c7c-9a26-4833-9587-55a139f413ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.116990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd0c7c-9a26-4833-9587-55a139f413ec admin admin] PATH_INFO: `/v3/projects/31886a7e34804ed2b81fd1d02a416f47/users/2619586afc25457bbc803aa82d0baf4a/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.120973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fd371-3539-4673-9568-fd31a7e53d6e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.127871 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.128558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.128840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.129118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.129894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.131894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.139498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd0c7c-9a26-4833-9587-55a139f413ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=31886a7e34804ed2b81fd1d02a416f47, user_id=2619586afc25457bbc803aa82d0baf4a, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.142477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd0c7c-9a26-4833-9587-55a139f413ec admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.158116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 997/2023] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:42 2022] PUT /identity/v3/projects/31886a7e34804ed2b81fd1d02a416f47/users/2619586afc25457bbc803aa82d0baf4a/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:42.168808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.172293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1027/2024] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:42 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:42.172504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.172504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.175079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.238969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] MFA Rules not processed for user `2619586afc25457bbc803aa82d0baf4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:42.240413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.327487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94fe77-a478-4559-9d99-24f3ab7f6249 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.341742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.342751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.343377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.345333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.347994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.357745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.357928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:42.416058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1028/2025] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:42 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:42.416058 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.418037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.422895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 998/2026] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:42.424175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.425949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.439653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a166c3d-0492-485e-bf24-5acd8630bcd1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.441930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a431e-81bb-4b88-b2b8-9e56735a75aa None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.446425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.447419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.447774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.447890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.448135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] PATH_INFO: `/v3/projects/112163d87ab9416481ca3fcc2e5baa3f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.449478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.449585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=112163d87ab9416481ca3fcc2e5baa3f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.449768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.450044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] PATH_INFO: `/v3/users/82c5ed38278e4b2ab8a6a0f1a6272ba6` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.451790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.452262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c5ed38278e4b2ab8a6a0f1a6272ba6)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.455351 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.459620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:42.529242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112163d87ab9416481ca3fcc2e5baa3f', 'request_id': 'req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.529422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c5ed38278e4b2ab8a6a0f1a6272ba6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.530038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c5ed38278e4b2ab8a6a0f1a6272ba6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.536267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112163d87ab9416481ca3fcc2e5baa3f', 'request_id': 'req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.539228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c5ed38278e4b2ab8a6a0f1a6272ba6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.539906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c5ed38278e4b2ab8a6a0f1a6272ba6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.543469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112163d87ab9416481ca3fcc2e5baa3f', 'request_id': 'req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.550279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c5ed38278e4b2ab8a6a0f1a6272ba6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.560747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] The token cache is being invalidate because project 112163d87ab9416481ca3fcc2e5baa3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:42.561346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.562065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.568440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1029/2027] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:42 2022] DELETE /identity/v3/projects/112163d87ab9416481ca3fcc2e5baa3f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:42.568440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.568440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.571757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 999/2028] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:42 2022] DELETE /identity/v3/users/82c5ed38278e4b2ab8a6a0f1a6272ba6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:42.572605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.574035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.648750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4450-7051-4fbc-9802-7439e0e75521 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.650922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dd7ca-95b3-41d9-ae3f-2d01783a52ed admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.652824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a167a1-00b2-4229-997e-3e4943fb393b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.653430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a167a1-00b2-4229-997e-3e4943fb393b service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.653711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a167a1-00b2-4229-997e-3e4943fb393b service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.653987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a167a1-00b2-4229-997e-3e4943fb393b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.656162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-97a167a1-00b2-4229-997e-3e4943fb393b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.722361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15bf54-643f-49e0-9bc8-9e3158a7e361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.723377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15bf54-643f-49e0-9bc8-9e3158a7e361 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.723796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15bf54-643f-49e0-9bc8-9e3158a7e361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.724327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15bf54-643f-49e0-9bc8-9e3158a7e361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.725368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15bf54-643f-49e0-9bc8-9e3158a7e361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.727977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15bf54-643f-49e0-9bc8-9e3158a7e361 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.735715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a167a1-00b2-4229-997e-3e4943fb393b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.737505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a167a1-00b2-4229-997e-3e4943fb393b service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.742581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1030/2029] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:42.742581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.742581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.742581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.747723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1000/2030] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:42.749071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6feb9a-159f-45cc-abe5-6cbd5ea89fb0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.749362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6feb9a-159f-45cc-abe5-6cbd5ea89fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.749643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6feb9a-159f-45cc-abe5-6cbd5ea89fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.756189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:42.785180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:42.785531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cc6feb9a-159f-45cc-abe5-6cbd5ea89fb0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:42.786672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.786816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc6feb9a-159f-45cc-abe5-6cbd5ea89fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.794996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1001/2031] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:42.798944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6feb9a-159f-45cc-abe5-6cbd5ea89fb0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.800517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1031/2032] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:42.801372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.809042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.813618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6feb9a-159f-45cc-abe5-6cbd5ea89fb0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.819529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee0e93-c672-4151-b2b4-c4fa8beaf4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.820290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee0e93-c672-4151-b2b4-c4fa8beaf4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.820710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee0e93-c672-4151-b2b4-c4fa8beaf4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.821060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee0e93-c672-4151-b2b4-c4fa8beaf4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.821856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee0e93-c672-4151-b2b4-c4fa8beaf4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.821950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1b539-9497-4b2e-9c4f-ae3869e7546b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:42.823541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee0e93-c672-4151-b2b4-c4fa8beaf4fd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.826716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:42.827584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.827859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.828121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] PATH_INFO: `/v3/users/2619586afc25457bbc803aa82d0baf4a` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.829861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2619586afc25457bbc803aa82d0baf4a)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:42.831742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:42.836592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1002/2033] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:42.837976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:42.838338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:42.838623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:42.859710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:42.861780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.875188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1003/2034] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:42.876672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.878637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.879679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:42.908997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2619586afc25457bbc803aa82d0baf4a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.909722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2619586afc25457bbc803aa82d0baf4a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.910339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2619586afc25457bbc803aa82d0baf4a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.919559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2619586afc25457bbc803aa82d0baf4a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.939926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2619586afc25457bbc803aa82d0baf4a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:42.958290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1032/2035] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:42 2022] DELETE /identity/v3/users/2619586afc25457bbc803aa82d0baf4a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:42.960607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:42.962417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:42.966023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86b3e0-e390-426f-a6e1-ac3e79b737a8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.037221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.038225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.038511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.038786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.040766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:43.052908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f2b6c-1d24-46b2-8b9e-7808b2437c07 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.057415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.058391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5d24c-226b-4f56-aa00-1292fd9286f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.058982 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.059478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5d24c-226b-4f56-aa00-1292fd9286f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.059901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5d24c-226b-4f56-aa00-1292fd9286f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.060275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5d24c-226b-4f56-aa00-1292fd9286f4 admin admin] PATH_INFO: `/v3/projects/44a1ab295b3a423aa2b3183a3e25ec0b/users/c26cd29ec472479a9a36ac9bb1f05580/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.072057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1004/2036] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:43.072706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.074412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:43.087343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312ac35-2a14-4b6c-90ad-5395445a2557 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.109675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518d11c-2adc-40b9-b1b4-3239aabb7e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.110969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5d24c-226b-4f56-aa00-1292fd9286f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a1ab295b3a423aa2b3183a3e25ec0b, user_id=c26cd29ec472479a9a36ac9bb1f05580, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.114512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518d11c-2adc-40b9-b1b4-3239aabb7e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.114908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518d11c-2adc-40b9-b1b4-3239aabb7e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.116781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518d11c-2adc-40b9-b1b4-3239aabb7e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.117828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518d11c-2adc-40b9-b1b4-3239aabb7e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:43.118239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518d11c-2adc-40b9-b1b4-3239aabb7e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.134309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5d24c-226b-4f56-aa00-1292fd9286f4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.139353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518d11c-2adc-40b9-b1b4-3239aabb7e17 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.166284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1033/2037] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:42 2022] PUT /identity/v3/projects/44a1ab295b3a423aa2b3183a3e25ec0b/users/c26cd29ec472479a9a36ac9bb1f05580/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:43.177192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd4e7-eb79-46ca-a52e-1f8e12d94c68 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.179786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd4e7-eb79-46ca-a52e-1f8e12d94c68 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.180818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd4e7-eb79-46ca-a52e-1f8e12d94c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.186844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1005/2038] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:43.196239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7025d6-d7c5-43f0-a1dc-528c3b6e44ab None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.196689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7025d6-d7c5-43f0-a1dc-528c3b6e44ab None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.196955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7025d6-d7c5-43f0-a1dc-528c3b6e44ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.207266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-775cd4e7-eb79-46ca-a52e-1f8e12d94c68 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:43.209023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-775cd4e7-eb79-46ca-a52e-1f8e12d94c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:43.251053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4f7025d6-d7c5-43f0-a1dc-528c3b6e44ab None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:43.252628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4f7025d6-d7c5-43f0-a1dc-528c3b6e44ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:43.294504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1034/2039] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:43.295687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd4e7-eb79-46ca-a52e-1f8e12d94c68 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.309203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd4e7-eb79-46ca-a52e-1f8e12d94c68 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.314740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.315609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.316067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.316475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.317376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.319725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.364144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1035/2040] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:43 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:43.365470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.366626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1006/2041] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:43.370967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7025d6-d7c5-43f0-a1dc-528c3b6e44ab None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.371195 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:43.379702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:43.386835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7025d6-d7c5-43f0-a1dc-528c3b6e44ab None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.429162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:43.506566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.507559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.507953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.508725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.509551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.512752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.537128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00045fd6-a871-4971-9b03-4fe359a1eb43 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.545309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.548907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.550554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.550992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.556923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1007/2042] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:43 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:43.558018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.561263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:43.576146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.579813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.585092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a9999-a30e-46a6-b068-0b8ef2139ca9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.589088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.590089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.590089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.590551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.611054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1036/2043] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:43.612341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.618390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.619876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.626409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9358f-1c27-447a-be4b-282276636e78 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.632214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.633493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.633927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.634346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.635384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.635542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1008/2044] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:43.637329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.649306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1037/2045] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:43.652724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.663605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db936b0e-54a3-4c72-997d-02ceeb55db24 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.669076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca002b-8f4b-4e9b-86b8-02335e84bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.671193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca002b-8f4b-4e9b-86b8-02335e84bc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.671492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca002b-8f4b-4e9b-86b8-02335e84bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.672153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca002b-8f4b-4e9b-86b8-02335e84bc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.674606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca002b-8f4b-4e9b-86b8-02335e84bc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.676287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca002b-8f4b-4e9b-86b8-02335e84bc45 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.690778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1009/2046] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:43.694740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:43.708796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea890-9908-4b43-86e4-7f56b792d7e8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:43.714614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:43.717013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.717304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.717589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] PATH_INFO: `/v3/projects/123e03a747e14903b3448a1a71be66b5/users/c31c3a1423b94e6398020c6981ccbb98/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.740542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123e03a747e14903b3448a1a71be66b5, user_id=c31c3a1423b94e6398020c6981ccbb98, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:43.743362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:43.765535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1038/2047] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:43 2022] PUT /identity/v3/projects/123e03a747e14903b3448a1a71be66b5/users/c31c3a1423b94e6398020c6981ccbb98/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:43.776742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa35e01-5ab7-4c3d-aaae-e736c00486e2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:43.776990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa35e01-5ab7-4c3d-aaae-e736c00486e2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:43.777267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa35e01-5ab7-4c3d-aaae-e736c00486e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:43.815825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4fa35e01-5ab7-4c3d-aaae-e736c00486e2 None None] MFA Rules not processed for user `c31c3a1423b94e6398020c6981ccbb98`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:43.817121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4fa35e01-5ab7-4c3d-aaae-e736c00486e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:43.920498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1010/2048] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:43.925941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.006118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd9372-f8f4-4039-bedf-9ba11bdb76b4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.012569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.013238 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.013543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.013836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.014554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.016579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.066124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1039/2049] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:43 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:44.068413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa35e01-5ab7-4c3d-aaae-e736c00486e2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.085470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa35e01-5ab7-4c3d-aaae-e736c00486e2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.176727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.177630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.179093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.179441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.180153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.182152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.210587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1011/2050] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:44 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:44.212311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.224981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c43dcb-d75a-4e11-829f-29d9bbb35b92 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.230788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.231577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.232599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.232953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.233691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.235611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.261737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1040/2051] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:44.264582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.286263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79f67e-58d1-4d07-8776-200502a63c9f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.292298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.293458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.294128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.294599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] PATH_INFO: `/v3/projects/00bb244e13754067847a352aa1ca0eb8/users/028d55a3fd644d8cbb7126d5fd9a1c13/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.306139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:44.319215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.321098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:44.339506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a2445-b33b-461f-a6e2-02758b5ba0e6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.345982 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00bb244e13754067847a352aa1ca0eb8, user_id=028d55a3fd644d8cbb7126d5fd9a1c13, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.359302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.361025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.361883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.362159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.362434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] PATH_INFO: `/v3/projects/c3e239184fd748e09c59e5e031ce3e06` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.363630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e239184fd748e09c59e5e031ce3e06)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.366265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.406783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1012/2052] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:44 2022] PUT /identity/v3/projects/00bb244e13754067847a352aa1ca0eb8/users/028d55a3fd644d8cbb7126d5fd9a1c13/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:44.408338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.462470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:44.483318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e239184fd748e09c59e5e031ce3e06', 'request_id': 'req-e5e66ae9-dec3-497e-8067-e8d937c73009'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.498210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e239184fd748e09c59e5e031ce3e06', 'request_id': 'req-e5e66ae9-dec3-497e-8067-e8d937c73009'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.514483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e239184fd748e09c59e5e031ce3e06', 'request_id': 'req-e5e66ae9-dec3-497e-8067-e8d937c73009'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.550352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dda5-e432-494a-ab5f-12d4c258457d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.568435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] The token cache is being invalidate because project c3e239184fd748e09c59e5e031ce3e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:44.570817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.574298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.583595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1041/2053] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:44 2022] DELETE /identity/v3/projects/c3e239184fd748e09c59e5e031ce3e06 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:44.585521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.590335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:44.693140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.694984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.695271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.698803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.698803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.702248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.733104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e66ae9-dec3-497e-8067-e8d937c73009 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.740109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.741342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.742386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1013/2054] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:44.742917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.743658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] PATH_INFO: `/v3/users/c2d4eb488aeb44928c1b06e754229e1e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.745148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d4eb488aeb44928c1b06e754229e1e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.746734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:44.747747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.749113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:44.779721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a1087-5d46-4c88-a29e-4df2ed990830 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:44.791717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a38450-08f1-40ba-bf70-14793ac05838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:44.793227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a38450-08f1-40ba-bf70-14793ac05838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.793675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a38450-08f1-40ba-bf70-14793ac05838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.794411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a38450-08f1-40ba-bf70-14793ac05838 admin admin] PATH_INFO: `/v3/projects/00bb244e13754067847a352aa1ca0eb8/users/028d55a3fd644d8cbb7126d5fd9a1c13/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.797001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a38450-08f1-40ba-bf70-14793ac05838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00bb244e13754067847a352aa1ca0eb8, user_id=028d55a3fd644d8cbb7126d5fd9a1c13, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:44.800468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a38450-08f1-40ba-bf70-14793ac05838 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:44.824748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d4eb488aeb44928c1b06e754229e1e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.825411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d4eb488aeb44928c1b06e754229e1e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.828450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d4eb488aeb44928c1b06e754229e1e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.841088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1014/2055] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:44 2022] PUT /identity/v3/projects/00bb244e13754067847a352aa1ca0eb8/users/028d55a3fd644d8cbb7126d5fd9a1c13/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:44.843732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:44.843913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:44.844196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:44.848163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d4eb488aeb44928c1b06e754229e1e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.876216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d4eb488aeb44928c1b06e754229e1e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:44.893200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1042/2056] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:44 2022] DELETE /identity/v3/users/c2d4eb488aeb44928c1b06e754229e1e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:44.895064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] MFA Rules not processed for user `028d55a3fd644d8cbb7126d5fd9a1c13`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:44.898293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:44.993221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:45.004735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1015/2057] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:45.013095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:45.017162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:45.125275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:45.137528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf49683-1800-4d55-b203-688d36078eea admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:45.151279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:45.152333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:45.152849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:45.153320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:45.171243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:45.173041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:45.184437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1043/2058] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:45.307714 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:45.309834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:45.324213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d585c-6b31-465a-b007-fc18bdf6793d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:45.328070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590f4a1-edf1-432b-8d2e-07c01b93db5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:45.329237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f4a1-edf1-432b-8d2e-07c01b93db5b service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:45.329697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f4a1-edf1-432b-8d2e-07c01b93db5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:45.330107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f4a1-edf1-432b-8d2e-07c01b93db5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:45.332471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a590f4a1-edf1-432b-8d2e-07c01b93db5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:45.414221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590f4a1-edf1-432b-8d2e-07c01b93db5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:45.415708 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590f4a1-edf1-432b-8d2e-07c01b93db5b service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:45.429189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1016/2059] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:45.503410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:45.505251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:45.660095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3ec6b-b9b4-49d0-8116-d2d3e27e881f service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:45.667208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:45.668709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:45.669178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:45.669971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] PATH_INFO: `/v3/projects/31886a7e34804ed2b81fd1d02a416f47` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:45.671517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31886a7e34804ed2b81fd1d02a416f47)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:45.675628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:45.746422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31886a7e34804ed2b81fd1d02a416f47', 'request_id': 'req-21370414-38de-4f6e-a575-c2b66fc84463'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:45.754886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31886a7e34804ed2b81fd1d02a416f47', 'request_id': 'req-21370414-38de-4f6e-a575-c2b66fc84463'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:45.763840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31886a7e34804ed2b81fd1d02a416f47', 'request_id': 'req-21370414-38de-4f6e-a575-c2b66fc84463'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:45.787044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] The token cache is being invalidate because project 31886a7e34804ed2b81fd1d02a416f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:45.787342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:45.788161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-21370414-38de-4f6e-a575-c2b66fc84463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:45.791429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1044/2060] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:45 2022] DELETE /identity/v3/projects/31886a7e34804ed2b81fd1d02a416f47 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:45.795764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd9a7e-2369-4915-8d77-e72d7ff6d082 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:45.796660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd9a7e-2369-4915-8d77-e72d7ff6d082 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:45.796956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd9a7e-2369-4915-8d77-e72d7ff6d082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:45.823754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-81bd9a7e-2369-4915-8d77-e72d7ff6d082 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:45.826317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-81bd9a7e-2369-4915-8d77-e72d7ff6d082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:45.929504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1017/2061] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:45.931607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316aeae-0d7c-4e2a-91d6-1756ee0ec6b8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:45.932030 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316aeae-0d7c-4e2a-91d6-1756ee0ec6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:45.932738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316aeae-0d7c-4e2a-91d6-1756ee0ec6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:45.962487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d316aeae-0d7c-4e2a-91d6-1756ee0ec6b8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:45.963732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d316aeae-0d7c-4e2a-91d6-1756ee0ec6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:45.976112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1045/2062] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:45.980762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd9a7e-2369-4915-8d77-e72d7ff6d082 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:45.991933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd9a7e-2369-4915-8d77-e72d7ff6d082 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:45.996723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:45.997394 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:45.997683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:45.998037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:45.998775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:45.999086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:46.001537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:46.017093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1018/2063] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:46.021062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316aeae-0d7c-4e2a-91d6-1756ee0ec6b8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:46.033809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316aeae-0d7c-4e2a-91d6-1756ee0ec6b8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:46.039680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:46.040368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.040699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.040981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.041744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:46.043905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:46.098374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1046/2064] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:46 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:46.102420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:46.115596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e4b43-fb92-456d-9f37-6d973922e5d7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:46.254133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:46.254948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.255453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.255793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.256718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:46.258826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:46.301439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1019/2065] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:46 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:46.305488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:46.316546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa94b-48cf-4fd8-a2da-daf61f36e302 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:46.331741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:46.332480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.332768 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.333047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.333785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:46.335395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:46.363823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1047/2066] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:46.368934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:46.381633 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce32d0e-bf80-4739-b973-6bf72c45fa0a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:46.399813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072766-e45b-46ce-bd3a-b1e879fe742d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:46.400843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072766-e45b-46ce-bd3a-b1e879fe742d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.401437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072766-e45b-46ce-bd3a-b1e879fe742d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.401910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072766-e45b-46ce-bd3a-b1e879fe742d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.403253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072766-e45b-46ce-bd3a-b1e879fe742d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:46.405160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072766-e45b-46ce-bd3a-b1e879fe742d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:46.417998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1020/2067] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:46.422184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:46.432988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6cd1-24bf-4e4f-9ba3-a9dd76be5178 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:46.452686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d315ba-4eb0-4b33-b60a-6092cc978e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:46.453595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d315ba-4eb0-4b33-b60a-6092cc978e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.454620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d315ba-4eb0-4b33-b60a-6092cc978e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.454963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d315ba-4eb0-4b33-b60a-6092cc978e5d admin admin] PATH_INFO: `/v3/projects/4a4f794ed2ae4e90a3f3c286e9fdca80/users/4dfe7e670b3d411089810ed88a740b46/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.488203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef80e-0c91-4043-9523-9ae70cf01105 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.488895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef80e-0c91-4043-9523-9ae70cf01105 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.489402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef80e-0c91-4043-9523-9ae70cf01105 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.491166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1021/2068] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 15:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 15:59:46.506464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d315ba-4eb0-4b33-b60a-6092cc978e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4f794ed2ae4e90a3f3c286e9fdca80, user_id=4dfe7e670b3d411089810ed88a740b46, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:46.522470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.522758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.523044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.531957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d315ba-4eb0-4b33-b60a-6092cc978e5d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:46.535026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:46.561470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:46.562721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:46.568689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1048/2069] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:46 2022] PUT /identity/v3/projects/4a4f794ed2ae4e90a3f3c286e9fdca80/users/4dfe7e670b3d411089810ed88a740b46/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:46.579206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.579705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.580290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.641969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] MFA Rules not processed for user `4dfe7e670b3d411089810ed88a740b46`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:46.647441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:46.669432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:46.728580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1022/2070] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 15:59:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:46.762634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:46.764119 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:46.788568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c118-56e1-4a32-99d7-6520a720cf1f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:46.801360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1049/2071] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:46.807341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:46.808112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.808431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.808746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:46.814159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:46.815631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:46.838647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:46.845432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:46.874485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1023/2072] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:46.972321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9ae23-acae-439a-ab0b-6c644c7f4a0a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:46.980121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725b679-0a34-4eb0-a5ca-8b1a2e2f54c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:46.980819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b679-0a34-4eb0-a5ca-8b1a2e2f54c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:46.981102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b679-0a34-4eb0-a5ca-8b1a2e2f54c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:46.981627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b679-0a34-4eb0-a5ca-8b1a2e2f54c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:47.037616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725b679-0a34-4eb0-a5ca-8b1a2e2f54c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:47.039140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725b679-0a34-4eb0-a5ca-8b1a2e2f54c5 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:47.057131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1050/2073] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:47.153500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:47.155246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:47.314668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caf748-7498-4cae-8e47-497d1d52f139 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:47.322662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:47.323911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:47.324953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:47.325545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] PATH_INFO: `/v3/projects/65947b4c83b84514be2978453090cfa2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:47.327214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65947b4c83b84514be2978453090cfa2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:47.330009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:47.493268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65947b4c83b84514be2978453090cfa2', 'request_id': 'req-ed4efccb-3e09-49ba-ae87-cfaab89b8021'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:47.509775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65947b4c83b84514be2978453090cfa2', 'request_id': 'req-ed4efccb-3e09-49ba-ae87-cfaab89b8021'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:47.537680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65947b4c83b84514be2978453090cfa2', 'request_id': 'req-ed4efccb-3e09-49ba-ae87-cfaab89b8021'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:47.569557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] The token cache is being invalidate because project 65947b4c83b84514be2978453090cfa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:47.571478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:47.572264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ed4efccb-3e09-49ba-ae87-cfaab89b8021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:47.574398 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1024/2074] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:47 2022] DELETE /identity/v3/projects/65947b4c83b84514be2978453090cfa2 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:47.582734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27f1ed-332d-4780-b9dc-47073873d329 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:47.583212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27f1ed-332d-4780-b9dc-47073873d329 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:47.583703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27f1ed-332d-4780-b9dc-47073873d329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:47.648202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9e27f1ed-332d-4780-b9dc-47073873d329 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:47.661352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9e27f1ed-332d-4780-b9dc-47073873d329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:47.863962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1051/2075] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:47.871487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c80e6-3769-4634-a63f-e296ec5b6504 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:47.871781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c80e6-3769-4634-a63f-e296ec5b6504 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:47.872124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c80e6-3769-4634-a63f-e296ec5b6504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:47.909308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bc3c80e6-3769-4634-a63f-e296ec5b6504 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:47.913490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bc3c80e6-3769-4634-a63f-e296ec5b6504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:47.929887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1025/2076] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:47.931140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27f1ed-332d-4780-b9dc-47073873d329 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:47.941752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27f1ed-332d-4780-b9dc-47073873d329 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:47.959630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:47.960324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:47.960635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:47.960911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:47.962205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:47.962515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:47.967846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:47.999578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1052/2077] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:48.004347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c80e6-3769-4634-a63f-e296ec5b6504 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:48.020430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c80e6-3769-4634-a63f-e296ec5b6504 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:48.031302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:48.032712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.033010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.033291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.034168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:48.037354 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:48.051454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:48.094135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1026/2078] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:48 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:48.100517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:48.120470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf47f79-a110-4895-bb2d-bfc9d1112a74 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:48.200922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:48.235991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:48.237420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.237821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.238203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.239521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:48.243302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:48.282355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1053/2079] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:48 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:48.287482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:48.304622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643729c-9aca-42c5-87b1-4b0bc5fb42e9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:48.311204 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:48.312214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.312702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.314480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.315548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:48.317719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:48.333280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1027/2080] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:48.337519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:48.356215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b48ff-246b-4ed0-bd88-4f0b295d3746 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:48.362678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d12bd-3c12-4d23-bc7c-7e32a67aa0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:48.364515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d12bd-3c12-4d23-bc7c-7e32a67aa0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.365193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d12bd-3c12-4d23-bc7c-7e32a67aa0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.365654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d12bd-3c12-4d23-bc7c-7e32a67aa0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.366714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d12bd-3c12-4d23-bc7c-7e32a67aa0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:48.369502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d12bd-3c12-4d23-bc7c-7e32a67aa0a1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:48.388243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1054/2081] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:48.391280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:48.414420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bd023-c6c0-4a16-9825-8bbf5db639b9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:48.421276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:48.424227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.424768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.425237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] PATH_INFO: `/v3/projects/d88460152126478b9080e3e0ad7a8105/users/b888399d72214df58837ab2c3386f679/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.451891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88460152126478b9080e3e0ad7a8105, user_id=b888399d72214df58837ab2c3386f679, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:48.456060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:48.489833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1028/2082] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:48 2022] PUT /identity/v3/projects/d88460152126478b9080e3e0ad7a8105/users/b888399d72214df58837ab2c3386f679/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:48.503507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.505603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.507919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.614423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] MFA Rules not processed for user `b888399d72214df58837ab2c3386f679`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:48.615752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:48.633495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:48.635427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:48.793553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdde9e-a0d3-4373-93aa-1a33c73a1b48 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:48.796153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1055/2083] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:48.797157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:48.799911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:48.809988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:48.810894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.811178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.811511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] PATH_INFO: `/v3/users/dfd3065e2d444dc6a4057713e738e0b4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.816563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd3065e2d444dc6a4057713e738e0b4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:48.820030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:48.832515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773cc6b-16fd-42ef-a24a-b9cc3e77119a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:48.849095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:48.850277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:48.850561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:48.850836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] PATH_INFO: `/v3/users/c31c3a1423b94e6398020c6981ccbb98` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:48.852642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31c3a1423b94e6398020c6981ccbb98)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:48.864969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:48.944278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd3065e2d444dc6a4057713e738e0b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:48.945162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd3065e2d444dc6a4057713e738e0b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:48.967836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd3065e2d444dc6a4057713e738e0b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:48.970262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd3065e2d444dc6a4057713e738e0b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:49.001028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31c3a1423b94e6398020c6981ccbb98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:49.001994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31c3a1423b94e6398020c6981ccbb98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:49.002838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31c3a1423b94e6398020c6981ccbb98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:49.006284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd3065e2d444dc6a4057713e738e0b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:49.042467 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31c3a1423b94e6398020c6981ccbb98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:49.083156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1029/2084] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:48 2022] DELETE /identity/v3/users/dfd3065e2d444dc6a4057713e738e0b4 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:49.083156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:49.091638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:49.101240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db011320-58f1-4aaf-92d8-301c96d521f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31c3a1423b94e6398020c6981ccbb98'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:49.161055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1056/2085] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:48 2022] DELETE /identity/v3/users/c31c3a1423b94e6398020c6981ccbb98 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:49.168926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.172935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.173584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.229581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:49.260280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:49.261970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:49.288514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:49.317417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7465f8-aeff-4304-b39b-3abee2907df7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:49.390176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1057/2086] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:49.419997 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:49.422160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:49.429759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:49.430919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.431492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.432604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.433687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:49.437634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:49.492436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1030/2087] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:49 2022] POST /identity/v3/projects => generated 377 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:49.495806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:49.498085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:49.528241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e99b1-266c-4a17-a132-445fbef60aa6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:49.602427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fadee-1be5-46d4-a89b-26ccdd72a76d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:49.611546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:49.612548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.613072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.613487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.613980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:49.614722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.615272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.615272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.616257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:49.619638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:49.631245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:49.633357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:49.648952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1058/2088] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:49.655561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1031/2089] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:49 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:49.656816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:49.673393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ec31-5371-4586-b7e4-32b0410740e6 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:49.683988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:49.685084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.685991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.686512 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.687626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:49.690928 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:49.721262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1059/2090] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:49.724992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:49.744869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22858eb-f2a5-45c0-be23-a80793680d05 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:49.771311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:49.777743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.778344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.780003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.781495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:49.784630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:49.804405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1032/2091] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:49.811635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:49.840545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb76a04-ae9b-409d-965a-283c1b7c63b0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:49.850808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29638914-4b8a-491b-a4a1-47a65fc79964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:49.852196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29638914-4b8a-491b-a4a1-47a65fc79964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.853396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:49.856612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:49.857129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29638914-4b8a-491b-a4a1-47a65fc79964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.857431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29638914-4b8a-491b-a4a1-47a65fc79964 admin admin] PATH_INFO: `/v3/projects/900b3b6e21f1435cb54e6b5575c0288d/users/0ed2be99c60e4cf89db0c4754fba62c5/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.879767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a0e26-6099-4060-a939-73e563d1ad88 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:49.893783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:49.894679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.895004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.895283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] PATH_INFO: `/v3/projects/0e7218a263864d068e82a5845e1fa739` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:49.896485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29638914-4b8a-491b-a4a1-47a65fc79964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900b3b6e21f1435cb54e6b5575c0288d, user_id=0ed2be99c60e4cf89db0c4754fba62c5, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:49.899329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e7218a263864d068e82a5845e1fa739)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:49.902048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:49.903192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29638914-4b8a-491b-a4a1-47a65fc79964 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:49.944402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1060/2092] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:49 2022] PUT /identity/v3/projects/900b3b6e21f1435cb54e6b5575c0288d/users/0ed2be99c60e4cf89db0c4754fba62c5/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:49.952497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:49.952817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:49.953088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.020867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:50.023412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] MFA Rules not processed for user `0ed2be99c60e4cf89db0c4754fba62c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:50.028283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.047436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7218a263864d068e82a5845e1fa739', 'request_id': 'req-69a6cf2d-6761-496f-8205-5bb17d75e706'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.086375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7218a263864d068e82a5845e1fa739', 'request_id': 'req-69a6cf2d-6761-496f-8205-5bb17d75e706'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.110296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7218a263864d068e82a5845e1fa739', 'request_id': 'req-69a6cf2d-6761-496f-8205-5bb17d75e706'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.152448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] The token cache is being invalidate because project 0e7218a263864d068e82a5845e1fa739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:50.154707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.155619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-69a6cf2d-6761-496f-8205-5bb17d75e706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.157818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1033/2093] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:49 2022] DELETE /identity/v3/projects/0e7218a263864d068e82a5845e1fa739 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:50.163857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee43c-07a0-4e01-9a49-8b26587a8923 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.164922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee43c-07a0-4e01-9a49-8b26587a8923 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.165292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee43c-07a0-4e01-9a49-8b26587a8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.184429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:50.214374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a29ee43c-07a0-4e01-9a49-8b26587a8923 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:50.215921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a29ee43c-07a0-4e01-9a49-8b26587a8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.276536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1061/2094] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:50.280912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:50.283056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.353769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1034/2095] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:50.355919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.360694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.362191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.416264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:50.420089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.443179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1035/2096] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:50.445451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:50.445598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9822436-839c-4513-bd3f-80e14a9ef623 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:50.446955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.452236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:50.453064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.453555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.453747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.454696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:50.461106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762d8a0-f56a-457b-84a2-2db4bcdc7234 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:50.463460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:50.471840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:50.482820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.483409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.484070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] PATH_INFO: `/v3/users/4dfe7e670b3d411089810ed88a740b46` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.499614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dfe7e670b3d411089810ed88a740b46)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:50.508427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:50.524797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1062/2097] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:50 2022] POST /identity/v3/projects => generated 375 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:50.526264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:50.620641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dfe7e670b3d411089810ed88a740b46'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.625264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dfe7e670b3d411089810ed88a740b46'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.628275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dcddf-634c-4ce2-9495-f3a85daa64a0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:50.638822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:50.642229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.645040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dfe7e670b3d411089810ed88a740b46'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.645161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.645479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.646265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:50.648653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:50.652277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dfe7e670b3d411089810ed88a740b46'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.653040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:50.674934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dfe7e670b3d411089810ed88a740b46'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.684409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1063/2098] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:50.684409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:50.690561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.727101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1036/2099] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:50 2022] DELETE /identity/v3/users/4dfe7e670b3d411089810ed88a740b46 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:50.727101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:50.728243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.730636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ace44e-3b28-4498-9f2f-cd7a2975d392 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:50.810976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:50.813840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:50.827002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:50.828075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.828219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.829463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.830014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:50.832489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:50.863351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1064/2100] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:50 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:50.864464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:50.866307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:50.879863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a92c47-f0c0-4efb-80e4-f2996516b112 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:50.883551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b658c-decb-4dd4-b4b1-fa6906f1d402 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:50.887035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcab20f-08b0-4c03-9193-92b72e72d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:50.887958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab20f-08b0-4c03-9193-92b72e72d10c admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.889550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab20f-08b0-4c03-9193-92b72e72d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.890019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab20f-08b0-4c03-9193-92b72e72d10c admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.892508 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:50.892855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcab20f-08b0-4c03-9193-92b72e72d10c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:50.895036 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcab20f-08b0-4c03-9193-92b72e72d10c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:50.898711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.899217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.899491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] PATH_INFO: `/v3/projects/123e03a747e14903b3448a1a71be66b5` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.901276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=123e03a747e14903b3448a1a71be66b5)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:50.903345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:50.963041 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1037/2101] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:50 2022] POST /identity/v3/projects => generated 391 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:50.966506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:50.966506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:50.966506 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:50.988751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123e03a747e14903b3448a1a71be66b5', 'request_id': 'req-7fc84737-f4bf-4705-9d25-716fdc41f3ea'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:50.999397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123e03a747e14903b3448a1a71be66b5', 'request_id': 'req-7fc84737-f4bf-4705-9d25-716fdc41f3ea'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.007373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:51.008891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.011901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123e03a747e14903b3448a1a71be66b5', 'request_id': 'req-7fc84737-f4bf-4705-9d25-716fdc41f3ea'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.033471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] The token cache is being invalidate because project 123e03a747e14903b3448a1a71be66b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:51.033774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.034668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.036391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1065/2102] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:50 2022] DELETE /identity/v3/projects/123e03a747e14903b3448a1a71be66b5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:51.037278 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.038711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.130955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84737-f4bf-4705-9d25-716fdc41f3ea admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.144255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.147223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.147530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.147812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.148692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.152183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.167763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1066/2103] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:51.173040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.175684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.180561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1038/2104] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:51.184528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.191616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.201776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efc6e-7b40-4295-bba7-4eb6f4a74621 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.209236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.210013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.210203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.210486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.211645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.213637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a460b-3f26-455d-940c-3b46bd49fcd9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.213867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.224717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.225911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.226187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.226460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] PATH_INFO: `/v3/users/028d55a3fd644d8cbb7126d5fd9a1c13` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.229209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=028d55a3fd644d8cbb7126d5fd9a1c13)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.231410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.264136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1067/2105] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:51 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:51.264762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.290207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1913-5f56-4291-8735-8f4332b50645 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.297159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.299576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.303186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.303577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] PATH_INFO: `/v3/projects/d823c2fbb3884e37a8eebf24c659996c/users/0723b0678983434489540456bb8b0e27/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.329922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d823c2fbb3884e37a8eebf24c659996c, user_id=0723b0678983434489540456bb8b0e27, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.332630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.342766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '028d55a3fd644d8cbb7126d5fd9a1c13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.343931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '028d55a3fd644d8cbb7126d5fd9a1c13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.362780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '028d55a3fd644d8cbb7126d5fd9a1c13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.362780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '028d55a3fd644d8cbb7126d5fd9a1c13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.371031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1068/2106] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:51 2022] PUT /identity/v3/projects/d823c2fbb3884e37a8eebf24c659996c/users/0723b0678983434489540456bb8b0e27/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:51.371031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.374429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.381010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '028d55a3fd644d8cbb7126d5fd9a1c13'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:51.407310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=85,keystone:INSERT=8,keystone:DELETE=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:51.427527 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1039/2107] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 15:59:51 2022] DELETE /identity/v3/users/028d55a3fd644d8cbb7126d5fd9a1c13 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:51.429687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.434990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.490946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe52bf-36a8-485c-ba3a-85a75f0b4f5f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.534344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:51.596156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cad0c-23c9-4326-b7f6-fad940c18864 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.605561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.607306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.607601 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.607933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.611597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.613957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.614655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.614949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.615222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.615961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.617617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.632180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.633852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.646288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1040/2108] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:51.647717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.650208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.654325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1069/2109] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:51.655165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.669774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1931314-c059-4129-a19c-28bc4d4dfcd0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.676175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.678368 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.678878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.679381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.679555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bda12-bcda-45ab-8d25-aedf8a8388d5 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.680532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.682583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.686418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.687165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.687497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.687923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.689558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.692178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.712396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1070/2110] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:51.713297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1041/2111] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:51.714869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.716184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:51.728467 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265becce-29d9-40bc-9def-4f2558455f08 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.731298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650a47-4961-4a31-9215-5f00c360a317 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:51.734289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcfd36-5e23-447d-8e75-022334042261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.736393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcfd36-5e23-447d-8e75-022334042261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.736819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcfd36-5e23-447d-8e75-022334042261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.737196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcfd36-5e23-447d-8e75-022334042261 admin admin] PATH_INFO: `/v3/projects/d823c2fbb3884e37a8eebf24c659996c/users/0723b0678983434489540456bb8b0e27/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.739619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcfd36-5e23-447d-8e75-022334042261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d823c2fbb3884e37a8eebf24c659996c, user_id=0723b0678983434489540456bb8b0e27, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.741700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce903ad-1bd9-4733-a1e2-34fef182c1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:51.743036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcfd36-5e23-447d-8e75-022334042261 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.743278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce903ad-1bd9-4733-a1e2-34fef182c1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.744428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce903ad-1bd9-4733-a1e2-34fef182c1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.744885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce903ad-1bd9-4733-a1e2-34fef182c1c8 admin admin] PATH_INFO: `/v3/projects/960d9a6b1f514344aa82bbada2b878ca/users/e9ca987fc23d4ff1a88dd4ed7b38980c/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.768846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1071/2112] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:51 2022] PUT /identity/v3/projects/d823c2fbb3884e37a8eebf24c659996c/users/0723b0678983434489540456bb8b0e27/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:51.774876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce903ad-1bd9-4733-a1e2-34fef182c1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=960d9a6b1f514344aa82bbada2b878ca, user_id=e9ca987fc23d4ff1a88dd4ed7b38980c, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:51.776462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.777073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.778291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.785156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce903ad-1bd9-4733-a1e2-34fef182c1c8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:51.831372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1042/2113] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:51 2022] PUT /identity/v3/projects/960d9a6b1f514344aa82bbada2b878ca/users/e9ca987fc23d4ff1a88dd4ed7b38980c/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:51.837627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:51.838892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:51.839645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:51.839890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] MFA Rules not processed for user `0723b0678983434489540456bb8b0e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:51.841571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.916916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:51.920452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:51.994591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1072/2114] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:52.007273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:52.008976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.110690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:52.114260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1043/2115] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:52.149522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae22028-c67d-4c9a-adc0-82bc5b2bc120 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:52.153654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:52.154343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.154840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.155135 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.163745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.198404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:52.200537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.280740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:52.284611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:52.300826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1073/2116] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:52.302365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:52.304735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.307888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ed95-479f-4c56-b737-eb9dd11211dc None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:52.316792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de609d9-2b35-4563-8616-df59bd908aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:52.319397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de609d9-2b35-4563-8616-df59bd908aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.319851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de609d9-2b35-4563-8616-df59bd908aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.320585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de609d9-2b35-4563-8616-df59bd908aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.336406 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476391fb-e9f8-4869-bc9f-61c0e90f1c83 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:52.343853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de609d9-2b35-4563-8616-df59bd908aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:52.348581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de609d9-2b35-4563-8616-df59bd908aa2 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:52.349125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:52.350004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.356697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.357025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] PATH_INFO: `/v3/projects/00bb244e13754067847a352aa1ca0eb8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.358962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00bb244e13754067847a352aa1ca0eb8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:52.367016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:52.376746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1044/2117] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 15:59:52.470614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00bb244e13754067847a352aa1ca0eb8', 'request_id': 'req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.487598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00bb244e13754067847a352aa1ca0eb8', 'request_id': 'req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.501747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00bb244e13754067847a352aa1ca0eb8', 'request_id': 'req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.521200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] The token cache is being invalidate because project 00bb244e13754067847a352aa1ca0eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:52.522191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.523432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.527236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1074/2118] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:52 2022] DELETE /identity/v3/projects/00bb244e13754067847a352aa1ca0eb8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:52.532743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ac79-5bf0-4301-9699-0867418618e7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.533040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ac79-5bf0-4301-9699-0867418618e7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.533323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ac79-5bf0-4301-9699-0867418618e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.553444 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:52.555224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.577337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2ec5ac79-5bf0-4301-9699-0867418618e7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:52.583191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2ec5ac79-5bf0-4301-9699-0867418618e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.651042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:52.697693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31ef13-58b5-4b01-8e84-70dc52c25ac0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:52.709420 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:52.709769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.710595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.710595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] PATH_INFO: `/v3/projects/4a4f794ed2ae4e90a3f3c286e9fdca80` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.724552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4f794ed2ae4e90a3f3c286e9fdca80)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:52.726569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:52.750157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1045/2119] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:52.756063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46939731-c01d-4f4b-8a5a-dcb48971d213 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.756865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46939731-c01d-4f4b-8a5a-dcb48971d213 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.758275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46939731-c01d-4f4b-8a5a-dcb48971d213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.762157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:52.796825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-46939731-c01d-4f4b-8a5a-dcb48971d213 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:52.798696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-46939731-c01d-4f4b-8a5a-dcb48971d213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.819443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4f794ed2ae4e90a3f3c286e9fdca80', 'request_id': 'req-e86754ba-615e-43f6-b834-7c2a2fc25cb3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.827785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4f794ed2ae4e90a3f3c286e9fdca80', 'request_id': 'req-e86754ba-615e-43f6-b834-7c2a2fc25cb3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.836675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4f794ed2ae4e90a3f3c286e9fdca80', 'request_id': 'req-e86754ba-615e-43f6-b834-7c2a2fc25cb3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.863077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] The token cache is being invalidate because project 4a4f794ed2ae4e90a3f3c286e9fdca80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 15:59:52.866331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.868069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e86754ba-615e-43f6-b834-7c2a2fc25cb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 15:59:52.873700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1075/2120] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 15:59:52 2022] DELETE /identity/v3/projects/4a4f794ed2ae4e90a3f3c286e9fdca80 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:52.879158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f202f2d-16b9-47f3-b7b4-b2506112bc19 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.879449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f202f2d-16b9-47f3-b7b4-b2506112bc19 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.879728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f202f2d-16b9-47f3-b7b4-b2506112bc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.903125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6f202f2d-16b9-47f3-b7b4-b2506112bc19 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:52.904624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6f202f2d-16b9-47f3-b7b4-b2506112bc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:52.912781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1046/2121] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:52.916517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46939731-c01d-4f4b-8a5a-dcb48971d213 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:52.918300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1076/2122] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:52.931864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46939731-c01d-4f4b-8a5a-dcb48971d213 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:52.938162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dfc94-c0bc-4aeb-8b95-20dc7f9e4af7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.938348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dfc94-c0bc-4aeb-8b95-20dc7f9e4af7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.940156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:52.941219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:52.941388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dfc94-c0bc-4aeb-8b95-20dc7f9e4af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.941532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:52.941669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:52.944121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:52.944267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:52.946846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:52.976463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1047/2123] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 15:59:52.977498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:52.996239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b41dfc94-c0bc-4aeb-8b95-20dc7f9e4af7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:52.997865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b41dfc94-c0bc-4aeb-8b95-20dc7f9e4af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:53.003914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc6aa-3060-4223-9944-a94619221c8b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.014524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.020439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1077/2124] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:53.021856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.023566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.024973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dfc94-c0bc-4aeb-8b95-20dc7f9e4af7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.025695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.026545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.034474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.050438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dfc94-c0bc-4aeb-8b95-20dc7f9e4af7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.058499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.060402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.060712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.061038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.062522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 15:59:53.062909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.065711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.090324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1078/2125] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 15:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 15:59:53.097573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.119065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1048/2126] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:52 2022] POST /identity/v3/projects => generated 409 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:53.120004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.123259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d5d5-af92-475d-89ca-6a98a78d3df6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.245458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.247631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.248515 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.249401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.250633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.253407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.254451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa509ab-84f2-4205-a835-549b8b6d6a7a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.265154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.265847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.266711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.266993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.267774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.269961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.291586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1079/2127] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:53 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:53.295685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.297052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1049/2128] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:53 2022] POST /identity/v3/users => generated 305 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:53.301510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.318825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85129a-6261-46d6-bea6-23571a0e7622 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.358794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:53.415422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef22c77-d9b5-4e25-9d7a-c7d91a3dfe60 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.421099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.423276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.423523 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.424323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.424551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.425039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.425782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.425905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.426776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.427682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.427860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.431629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.453002 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:53.454473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1050/2129] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:53.457076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.473464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1080/2130] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:53 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:53.477767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.478610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5ffa9-202f-42c3-88d6-a71cbfae27a0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.484480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.487131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.487582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.488155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.489232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.491237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.496047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb36bdf-4562-4e84-b269-ab1f95bbd966 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.502993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.503691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.504018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.504305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.505434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.507456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.509713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1051/2131] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:53.514458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.522291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1081/2132] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:53.523611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.532784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741eafe-37c0-4fd1-8ac6-35e8b0f7cac3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.538970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923849e-d150-4aae-82cb-13ea2ad89f0d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.539161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce380d-82b7-4172-9a9c-54c84b139474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.539910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce380d-82b7-4172-9a9c-54c84b139474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.540194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce380d-82b7-4172-9a9c-54c84b139474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.540555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce380d-82b7-4172-9a9c-54c84b139474 admin admin] PATH_INFO: `/v3/projects/67049ceef8fa4c33b05b8e7e167df78e/users/2003726ff00349cfb9b1ad4a706a3b04/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.545546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.546885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.548457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.549219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.550312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.552448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.562235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce380d-82b7-4172-9a9c-54c84b139474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67049ceef8fa4c33b05b8e7e167df78e, user_id=2003726ff00349cfb9b1ad4a706a3b04, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.565893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce380d-82b7-4172-9a9c-54c84b139474 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.570986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1082/2133] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:53.572201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.584071 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e84d-dce1-460e-b4d1-f9e8bde52830 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.587846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1052/2134] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:53 2022] PUT /identity/v3/projects/67049ceef8fa4c33b05b8e7e167df78e/users/2003726ff00349cfb9b1ad4a706a3b04/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:53.599898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9811f-575d-4e07-af5f-8589fd6a09d8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.600299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9811f-575d-4e07-af5f-8589fd6a09d8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.600733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9811f-575d-4e07-af5f-8589fd6a09d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.645747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-57d9811f-575d-4e07-af5f-8589fd6a09d8 None None] MFA Rules not processed for user `2003726ff00349cfb9b1ad4a706a3b04`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:53.647224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-57d9811f-575d-4e07-af5f-8589fd6a09d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:53.697010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d61d6-da17-45ce-91a3-748d468739d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.698006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d61d6-da17-45ce-91a3-748d468739d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.698358 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d61d6-da17-45ce-91a3-748d468739d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.698726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d61d6-da17-45ce-91a3-748d468739d1 admin admin] PATH_INFO: `/v3/projects/92e79292402548189e0f8f0d7de557ef/users/f062e303fea7439c9665fb11cf028fdb/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.722655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d61d6-da17-45ce-91a3-748d468739d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e79292402548189e0f8f0d7de557ef, user_id=f062e303fea7439c9665fb11cf028fdb, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.738244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d61d6-da17-45ce-91a3-748d468739d1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.739322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1053/2135] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 15:59:53.740565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9811f-575d-4e07-af5f-8589fd6a09d8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.759531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9811f-575d-4e07-af5f-8589fd6a09d8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.765738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.767857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.768267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.768727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.770026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.771545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1083/2136] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:53 2022] PUT /identity/v3/projects/92e79292402548189e0f8f0d7de557ef/users/f062e303fea7439c9665fb11cf028fdb/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:53.772943 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:53.786013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.786398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.786592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.826278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1054/2137] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:53 2022] POST /identity/v3/projects => generated 409 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:53.831019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] MFA Rules not processed for user `f062e303fea7439c9665fb11cf028fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:53.832538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:53.834434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.928008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1084/2138] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:53.928686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:53.930539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:53.952256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:53.959846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4d207-24d4-4144-a3a4-8a4381328750 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:53.967906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:53.968679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:53.968963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:53.969279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:53.970033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:53.972432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:54.013671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1055/2139] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:53 2022] POST /identity/v3/users => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:54.015476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:54.038307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:54.042186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b6b7-78c4-4846-a393-18282dcebb1e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:54.050219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:54.050890 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.051175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.051450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.053072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:54.054906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:54.056179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc384a-7dc8-4aa9-96fc-c0db15296142 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:54.062172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0206e09e4f426d9f4b54543d89c039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:54.064137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.064848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.065362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.067255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1056/2140] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 15:59:54.068343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:54.072600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:54.081653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:54.083911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:54.101069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1085/2141] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:54.102014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:54.124684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34dcd7-781e-44b1-8181-fc5701a378da service glance] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:54.131760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:54.133020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.133407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.133815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] PATH_INFO: `/v3/projects/948bf87e9eda475198987115059b575d/users/c03b5c9970764e20b9145300d24f92bd/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.160757 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=948bf87e9eda475198987115059b575d, user_id=c03b5c9970764e20b9145300d24f92bd, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:54.164466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:54.185717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dc34c-4a4f-409b-a905-bf522a53c9c8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:54.186886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1086/2142] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:54 2022] PUT /identity/v3/projects/948bf87e9eda475198987115059b575d/users/c03b5c9970764e20b9145300d24f92bd/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:54.194361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:54.329940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a568d1-0428-4494-9600-910478df14e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:54.330661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a568d1-0428-4494-9600-910478df14e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.330971 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a568d1-0428-4494-9600-910478df14e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.333678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a568d1-0428-4494-9600-910478df14e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.339619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-60a568d1-0428-4494-9600-910478df14e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:54.408853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cd023-cee8-45bb-976a-5f120a5fe43a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:54.432911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:54.433750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.434035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.434321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.437678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:54.440888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:54.464642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1087/2143] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:54.472696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:54.513452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6b498-b211-4a99-a7cf-fb687823a878 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:54.530760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84776c-2b25-422b-b0f0-8f759990f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:54.531654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84776c-2b25-422b-b0f0-8f759990f621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.531964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84776c-2b25-422b-b0f0-8f759990f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.533066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84776c-2b25-422b-b0f0-8f759990f621 admin admin] PATH_INFO: `/v3/projects/948bf87e9eda475198987115059b575d/users/c03b5c9970764e20b9145300d24f92bd/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.537863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84776c-2b25-422b-b0f0-8f759990f621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948bf87e9eda475198987115059b575d, user_id=c03b5c9970764e20b9145300d24f92bd, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:54.542570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84776c-2b25-422b-b0f0-8f759990f621 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:54.581285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1088/2144] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:54 2022] PUT /identity/v3/projects/948bf87e9eda475198987115059b575d/users/c03b5c9970764e20b9145300d24f92bd/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 15:59:54.581472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a568d1-0428-4494-9600-910478df14e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:54.582530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a568d1-0428-4494-9600-910478df14e2 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:54.582956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.583244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.583526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.633080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1057/2145] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:54.644413 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] MFA Rules not processed for user `c03b5c9970764e20b9145300d24f92bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:54.645646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:54.673837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:54.674209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:54.674975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:54.742085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] MFA Rules not processed for user `c26cd29ec472479a9a36ac9bb1f05580`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 15:59:54.744998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:54.824130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:54.845743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1089/2146] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:54 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:54.895916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1058/2147] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 15:59:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 15:59:54.900754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:54.903442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:54.961937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:54.965673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:54.991433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:55.035016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca6806-71b5-4fb1-8a15-849071846562 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.040318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.041001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.041310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:55.041547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:55.042319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:55.044487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:55.090000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1090/2148] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 15:59:54 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 15:59:55.092094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:55.159972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc9e48-902e-4cb0-ada0-b9fbbc819466 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.245141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e055e2-7fc3-482f-8319-56cae51a1377 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.248659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.249388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.249681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:55.249954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:55.259374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.260787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.261069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:55.261343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:55.263941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:55.269511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:55.324055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1091/2149] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 15:59:55 2022] POST /identity/v3/users => generated 296 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 15:59:55.328274 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:55.341517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:55.344039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:55.355663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2237-8e5c-4a12-94ef-5c7806e033f7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.367317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.368069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.368473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:55.368691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:55.369368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:55.370972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:55.373352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1059/2150] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:54 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:55.400455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1092/2151] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:55.403831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:55.419033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c1d14-e684-45a6-8466-1cb8f8928407 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.445148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.446061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.446341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:55.446617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] PATH_INFO: `/v3/projects/34304975d4e94e8eabf8794ccd98e2fd/users/4fc158cd498a450a82248cba8a3a3eac/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:55.482420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34304975d4e94e8eabf8794ccd98e2fd, user_id=4fc158cd498a450a82248cba8a3a3eac, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:55.488233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:55.517054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1060/2152] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:55 2022] PUT /identity/v3/projects/34304975d4e94e8eabf8794ccd98e2fd/users/4fc158cd498a450a82248cba8a3a3eac/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:55.525468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:55.558443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38c6f9-a75b-400e-aa56-5dfcffd60583 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.734925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:55.755704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.756493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.756794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:55.757186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:55.759482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:55.762545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:55.775553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:55.779063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1093/2153] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 15:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 15:59:55.787902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:55.816621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21937f7b-f99c-4dfe-9559-3d4441b9d2fa admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.834537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.838104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.838990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:55.839451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] PATH_INFO: `/v3/projects/34304975d4e94e8eabf8794ccd98e2fd/users/4fc158cd498a450a82248cba8a3a3eac/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:55.846137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34304975d4e94e8eabf8794ccd98e2fd, user_id=4fc158cd498a450a82248cba8a3a3eac, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:55.850051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:55.880648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1094/2154] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 15:59:55 2022] PUT /identity/v3/projects/34304975d4e94e8eabf8794ccd98e2fd/users/4fc158cd498a450a82248cba8a3a3eac/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 15:59:55.882006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:59:55.887854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56244f-87bb-4bfb-a0d0-1bd8bce29843 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.905271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:59:55.980376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f37238-53ed-4a55-8554-d13ec188b388 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:59:55.997822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.998198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576cd9b-7818-4cbd-a30b-465bb511b5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:59:55.999491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:55.999773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:56.000279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd9b-7818-4cbd-a30b-465bb511b5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:59:56.001164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd9b-7818-4cbd-a30b-465bb511b5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:59:56.001319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:56.002364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd9b-7818-4cbd-a30b-465bb511b5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:59:56.004983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:59:56.097097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576cd9b-7818-4cbd-a30b-465bb511b5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:56.099170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576cd9b-7818-4cbd-a30b-465bb511b5d3 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:56.111215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1061/2155] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:59:56.134637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:59:56.136424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:59:56.158774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1095/2156] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 15:59:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:01.591123 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f41873-9e72-42b9-b915-87c3e55bee2e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:01.592025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f41873-9e72-42b9-b915-87c3e55bee2e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:01.592657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f41873-9e72-42b9-b915-87c3e55bee2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:01.664496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d9f41873-9e72-42b9-b915-87c3e55bee2e None None] MFA Rules not processed for user `e9ca987fc23d4ff1a88dd4ed7b38980c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:01.665944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d9f41873-9e72-42b9-b915-87c3e55bee2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:01.701021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:01.702699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:01.815519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1062/2157] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:01.825709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:01.826275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:01.829493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:01.859485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70101dfe-8c62-4de9-baef-da5478e60f8b service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:01.872862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] MFA Rules not processed for user `e9ca987fc23d4ff1a88dd4ed7b38980c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:01.874187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:01.874322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:01.875766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:01.876061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:01.876341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] PATH_INFO: `/v3/users/b888399d72214df58837ab2c3386f679` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:01.878007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b888399d72214df58837ab2c3386f679)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:01.883027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1063/2158] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:01.884050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:01.897745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:01.899165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:01.923864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:02.028505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b888399d72214df58837ab2c3386f679'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:02.029157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b888399d72214df58837ab2c3386f679'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:02.029855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b888399d72214df58837ab2c3386f679'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:02.054438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b888399d72214df58837ab2c3386f679'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:02.090681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:02.093187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4c3b-52bc-4d93-b0b7-20a8e92e4083 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:02.099388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-bd6dd9c7-18d6-4e3f-8bba-3b99a17df75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b888399d72214df58837ab2c3386f679'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:02.102241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:02.103139 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:02.103830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:02.104954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:02.119682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:02.137836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:02.139694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:02.142611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1096/2159] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:00:01 2022] DELETE /identity/v3/users/b888399d72214df58837ab2c3386f679 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:00:02.149478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:02.162581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:02.163148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:02.175619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1064/2160] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:02.204136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:02.205107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:02.473589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1097/2161] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:02.496571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:02.498002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:02.639559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e979e-f643-4ff3-b4ae-15a86d9068bb service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:02.740176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:02.740951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:02.741238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:02.741515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:02.757256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:02.759457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:02.779739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1065/2162] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:03.551397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:03.555492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:03.577924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8fa56-bd5c-431c-b8a7-e9f4e0c19a9a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:03.598374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:03.600914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:03.601386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:03.601815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] PATH_INFO: `/v3/projects/d88460152126478b9080e3e0ad7a8105` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:03.613949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88460152126478b9080e3e0ad7a8105)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:03.616170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:03.692068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88460152126478b9080e3e0ad7a8105', 'request_id': 'req-d52f30cd-8338-40a6-bdaa-1c9aa9437533'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.699461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88460152126478b9080e3e0ad7a8105', 'request_id': 'req-d52f30cd-8338-40a6-bdaa-1c9aa9437533'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.709462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88460152126478b9080e3e0ad7a8105', 'request_id': 'req-d52f30cd-8338-40a6-bdaa-1c9aa9437533'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.728767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] The token cache is being invalidate because project d88460152126478b9080e3e0ad7a8105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:00:03.729819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.730548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d52f30cd-8338-40a6-bdaa-1c9aa9437533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.736906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1098/2163] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:00:03 2022] DELETE /identity/v3/projects/d88460152126478b9080e3e0ad7a8105 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:03.737969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:03.759213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e19b-4fb5-4fda-88b6-7ae62a484a8c service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:03.873697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:03.884334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:03.885362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:03.885712 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:03.886067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] PATH_INFO: `/v3/users/0ed2be99c60e4cf89db0c4754fba62c5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:03.887673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed2be99c60e4cf89db0c4754fba62c5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:03.889871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:03.977988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed2be99c60e4cf89db0c4754fba62c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.979612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed2be99c60e4cf89db0c4754fba62c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.988332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed2be99c60e4cf89db0c4754fba62c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:03.989603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed2be99c60e4cf89db0c4754fba62c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:04.001728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed2be99c60e4cf89db0c4754fba62c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:04.024804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a2f35-35be-4951-968d-bb5fb5732dd7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:04.025193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a2f35-35be-4951-968d-bb5fb5732dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:04.025899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a2f35-35be-4951-968d-bb5fb5732dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:04.071150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1066/2164] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:00:03 2022] DELETE /identity/v3/users/0ed2be99c60e4cf89db0c4754fba62c5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:04.080101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6a3a2f35-35be-4951-968d-bb5fb5732dd7 None None] MFA Rules not processed for user `e9ca987fc23d4ff1a88dd4ed7b38980c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:04.081599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6a3a2f35-35be-4951-968d-bb5fb5732dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:04.225785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1099/2165] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:04.663981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:04.665484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:04.849109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af354b7c-43e9-4986-a7fd-514a52573207 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:04.855010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:04.858089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:04.858606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:04.859045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] PATH_INFO: `/v3/projects/900b3b6e21f1435cb54e6b5575c0288d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:04.861261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=900b3b6e21f1435cb54e6b5575c0288d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:04.863295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:04.909360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:04.909360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:04.909360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:04.999392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900b3b6e21f1435cb54e6b5575c0288d', 'request_id': 'req-fd71d856-c829-4781-961a-0c08feae15b5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.009263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900b3b6e21f1435cb54e6b5575c0288d', 'request_id': 'req-fd71d856-c829-4781-961a-0c08feae15b5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.023461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] MFA Rules not processed for user `4fc158cd498a450a82248cba8a3a3eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:05.024930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:05.032335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900b3b6e21f1435cb54e6b5575c0288d', 'request_id': 'req-fd71d856-c829-4781-961a-0c08feae15b5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.038829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:05.054903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] The token cache is being invalidate because project 900b3b6e21f1435cb54e6b5575c0288d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:00:05.055577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.056595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.061439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1067/2166] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:00:04 2022] DELETE /identity/v3/projects/900b3b6e21f1435cb54e6b5575c0288d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:05.065391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:05.067226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:05.073443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:05.207591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1100/2167] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:00:05.219797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d856-c829-4781-961a-0c08feae15b5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:05.229922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:05.231238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:05.231370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:05.231459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:05.231459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] PATH_INFO: `/v3/users/0723b0678983434489540456bb8b0e27` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:05.232707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:05.240328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0723b0678983434489540456bb8b0e27)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:05.245232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:05.326412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0723b0678983434489540456bb8b0e27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.327482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0723b0678983434489540456bb8b0e27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.337975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0723b0678983434489540456bb8b0e27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.339476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0723b0678983434489540456bb8b0e27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.366139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd44fe-0c01-4418-b7b9-ca4e3dc2a8bd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:05.372448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0723b0678983434489540456bb8b0e27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:05.392000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:05.394750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:05.396556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:05.398768 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:05.410497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:05.462604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1068/2168] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:00:05 2022] DELETE /identity/v3/users/0723b0678983434489540456bb8b0e27 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:05.561642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:05.565856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:05.602897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1101/2169] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:06.381785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:06.383280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:06.567922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:06.574194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:06.609896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f9b58-c54c-4838-ac39-38ca27be5284 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:06.655786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:06.656776 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:06.657161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:06.657361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] PATH_INFO: `/v3/projects/d823c2fbb3884e37a8eebf24c659996c` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:06.665974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d19ec-dc2d-4063-b876-531732c73c1c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:06.670531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d823c2fbb3884e37a8eebf24c659996c)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:06.674272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:06.691454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:06.693029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:06.693029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:06.693317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] PATH_INFO: `/v3/users/2003726ff00349cfb9b1ad4a706a3b04` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:06.699121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2003726ff00349cfb9b1ad4a706a3b04)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:06.701140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:06.776038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:06.783414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:06.810388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd823c2fbb3884e37a8eebf24c659996c', 'request_id': 'req-a5815905-58d9-4d79-bc2c-6d3c04afcb04'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.838574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2003726ff00349cfb9b1ad4a706a3b04'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.839397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2003726ff00349cfb9b1ad4a706a3b04'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.843859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd823c2fbb3884e37a8eebf24c659996c', 'request_id': 'req-a5815905-58d9-4d79-bc2c-6d3c04afcb04'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.853976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd823c2fbb3884e37a8eebf24c659996c', 'request_id': 'req-a5815905-58d9-4d79-bc2c-6d3c04afcb04'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.858956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2003726ff00349cfb9b1ad4a706a3b04'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.860268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2003726ff00349cfb9b1ad4a706a3b04'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.874160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f9c6fcb-766a-4d2e-ab96-435564f72d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2003726ff00349cfb9b1ad4a706a3b04'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.887181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] The token cache is being invalidate because project d823c2fbb3884e37a8eebf24c659996c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:00:06.887857 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.888687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a5815905-58d9-4d79-bc2c-6d3c04afcb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:06.896229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1102/2170] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:00:06 2022] DELETE /identity/v3/projects/d823c2fbb3884e37a8eebf24c659996c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:06.898094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1154b-4ef8-4af9-9c0f-4c65029cb9fc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:06.898388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1154b-4ef8-4af9-9c0f-4c65029cb9fc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:06.898666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1154b-4ef8-4af9-9c0f-4c65029cb9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:06.920870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1069/2171] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:00:06 2022] DELETE /identity/v3/users/2003726ff00349cfb9b1ad4a706a3b04 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:00:06.925884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:06.926172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:06.926511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:06.933703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b7e1154b-4ef8-4af9-9c0f-4c65029cb9fc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:06.939051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b7e1154b-4ef8-4af9-9c0f-4c65029cb9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:06.959882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:06.962000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:07.112192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1070/2172] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:00:07.132386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1103/2173] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:07.133128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:07.133128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:07.223795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9318ada-a714-427e-a7be-c63d07985609 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.224084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9318ada-a714-427e-a7be-c63d07985609 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.224390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9318ada-a714-427e-a7be-c63d07985609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.265415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c9318ada-a714-427e-a7be-c63d07985609 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:07.268168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c9318ada-a714-427e-a7be-c63d07985609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:07.296521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1104/2174] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:00:07.298209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9318ada-a714-427e-a7be-c63d07985609 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:07.311658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9318ada-a714-427e-a7be-c63d07985609 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:07.323962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:07.324688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.325480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.325766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.326827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:00:07.326827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:07.349519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:07.362627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa03ecf-3a2e-4c8b-8596-0ceff2d44811 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:07.367710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:07.368560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.369083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.369459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.387838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1105/2175] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:00:07.390485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:07.392216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:07.393575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:07.429152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77f5d4-a190-4972-a24f-a7df6f3666fe admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:07.433733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1071/2176] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:00:07.438642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:07.439436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.439720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.439996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.440784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:07.442922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:07.521115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:07.527703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1106/2177] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:00:07 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:00:07.535145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8ed51-d552-4126-b4e0-d713bfe61693 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:07.656760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:07.700176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:07.701046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.701381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.701647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.703315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:07.706881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:07.761165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:07.767938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1072/2178] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:00:07 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:00:07.773672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77276d-df6a-4867-9b41-9bf17e48c5dd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:07.779578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:07.780603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.780603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.780811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.782534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:07.784319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:07.820018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:07.834479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1107/2179] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:00:07.836749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2d46c-06b3-4d3d-ade3-df5886a2c4ce admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:07.854100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67077508-549e-4f06-b769-68e598360dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:07.855770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077508-549e-4f06-b769-68e598360dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.856056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077508-549e-4f06-b769-68e598360dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.856335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077508-549e-4f06-b769-68e598360dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.857697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67077508-549e-4f06-b769-68e598360dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:07.859455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67077508-549e-4f06-b769-68e598360dd6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:07.883056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1073/2180] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:00:07.884046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:07.899649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd2cb-4249-4e2a-b53e-9291857137e6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:07.910527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:07.911442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.911733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.912012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] PATH_INFO: `/v3/projects/8bc2a18d64e440eabb0f4f4a072cf671/users/5c98ae10605b4af1b367a28870919e47/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:07.944165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc2a18d64e440eabb0f4f4a072cf671, user_id=5c98ae10605b4af1b367a28870919e47, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:07.948434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:07.971261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1108/2181] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:00:07 2022] PUT /identity/v3/projects/8bc2a18d64e440eabb0f4f4a072cf671/users/5c98ae10605b4af1b367a28870919e47/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:07.977510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:07.978191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:07.978191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:08.001984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:08.003347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:08.085224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1074/2182] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:08.096151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:08.178251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:08.190696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc32299-dc02-4230-97b7-68a1e370d0e3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:08.195437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:08.196106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:08.196743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:08.197175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:08.208628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:08.210078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:08.222519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1109/2183] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:08.994556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:08.999031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:09.019399 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11ac16-1786-4e0d-bc9e-e3665167bad1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:09.027688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:09.029456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:09.029908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:09.030336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] PATH_INFO: `/v3/projects/67049ceef8fa4c33b05b8e7e167df78e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:09.032767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=67049ceef8fa4c33b05b8e7e167df78e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:09.035200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:09.131674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67049ceef8fa4c33b05b8e7e167df78e', 'request_id': 'req-315da08b-2170-4c6d-a188-c58773ba6f9d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.154312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67049ceef8fa4c33b05b8e7e167df78e', 'request_id': 'req-315da08b-2170-4c6d-a188-c58773ba6f9d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.182286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67049ceef8fa4c33b05b8e7e167df78e', 'request_id': 'req-315da08b-2170-4c6d-a188-c58773ba6f9d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.204056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] The token cache is being invalidate because project 67049ceef8fa4c33b05b8e7e167df78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:00:09.204415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.205094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.209539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1075/2184] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:00:08 2022] DELETE /identity/v3/projects/67049ceef8fa4c33b05b8e7e167df78e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:09.210789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:09.233558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98967cf-0200-430b-8da7-4fc05d7b280a service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:09.392165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:09.393991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:09.394276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:09.397796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] PATH_INFO: `/v3/users/c03b5c9970764e20b9145300d24f92bd` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:09.402193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03b5c9970764e20b9145300d24f92bd)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:09.405942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:09.495087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03b5c9970764e20b9145300d24f92bd'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.496118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03b5c9970764e20b9145300d24f92bd'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.496943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03b5c9970764e20b9145300d24f92bd'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.505970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03b5c9970764e20b9145300d24f92bd'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.521517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-be9de561-a5c9-4c56-b473-1304110ec67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03b5c9970764e20b9145300d24f92bd'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:09.540877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1110/2185] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:00:09 2022] DELETE /identity/v3/users/c03b5c9970764e20b9145300d24f92bd => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:10.591549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:10.593168 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:10.639995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315da08b-2170-4c6d-a188-c58773ba6f9d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:10.798871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:10.861938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:10.862768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:10.863092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:10.863324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] PATH_INFO: `/v3/projects/948bf87e9eda475198987115059b575d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:10.864657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=948bf87e9eda475198987115059b575d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:10.866574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:10.994585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948bf87e9eda475198987115059b575d', 'request_id': 'req-fb860756-c588-48e0-97ca-c452bbe95865'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:11.009056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948bf87e9eda475198987115059b575d', 'request_id': 'req-fb860756-c588-48e0-97ca-c452bbe95865'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:11.035498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948bf87e9eda475198987115059b575d', 'request_id': 'req-fb860756-c588-48e0-97ca-c452bbe95865'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:11.063477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] The token cache is being invalidate because project 948bf87e9eda475198987115059b575d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:00:11.063782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:11.064504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:11.066595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1076/2186] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:00:10 2022] DELETE /identity/v3/projects/948bf87e9eda475198987115059b575d => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:15.351004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee322371-1f2f-4492-a2e1-59f7defd3517 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:15.351953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee322371-1f2f-4492-a2e1-59f7defd3517 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:15.352504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee322371-1f2f-4492-a2e1-59f7defd3517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:15.386469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ee322371-1f2f-4492-a2e1-59f7defd3517 None None] MFA Rules not processed for user `5c98ae10605b4af1b367a28870919e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:15.388630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ee322371-1f2f-4492-a2e1-59f7defd3517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:15.465815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:15.486109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1111/2187] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:15.490266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:15.492319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:15.585947 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb860756-c588-48e0-97ca-c452bbe95865 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:15.596387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:15.597349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:15.597822 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:15.598257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:15.599350 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:15.601633 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:15.679068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1077/2188] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:00:15 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:00:15.683290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee322371-1f2f-4492-a2e1-59f7defd3517 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:15.697331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee322371-1f2f-4492-a2e1-59f7defd3517 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:15.807576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:15.808399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:15.808647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:15.808925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:15.809845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:15.811920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:15.852914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:15.864717 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1112/2189] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:00:15 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:00:15.873535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c23e7-2273-4264-888d-d044edb6713b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:15.879205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:15.879999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:15.880291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:15.880597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:15.881338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:15.883216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:15.900582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1078/2190] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:00:15.904693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:15.916242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182426d-db34-431c-847f-f46c63c23adf admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:15.921344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:15.922899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:15.923321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:15.923640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] PATH_INFO: `/v3/projects/af77fdfaf6cb4ae18d33d3ae4bac9459/users/e929fc58e8e24db8b5ac4f7976b54320/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:15.944909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af77fdfaf6cb4ae18d33d3ae4bac9459, user_id=e929fc58e8e24db8b5ac4f7976b54320, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:15.947984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:15.964006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1113/2191] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:00:15 2022] PUT /identity/v3/projects/af77fdfaf6cb4ae18d33d3ae4bac9459/users/e929fc58e8e24db8b5ac4f7976b54320/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:15.967806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:15.980170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008cb51-c3c4-42e4-b333-4adbf474f8a6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:16.041113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:16.053097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3720fe-3594-4c27-ac51-07cb487d348b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:16.053771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3720fe-3594-4c27-ac51-07cb487d348b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:16.054056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3720fe-3594-4c27-ac51-07cb487d348b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:16.054333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3720fe-3594-4c27-ac51-07cb487d348b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:16.055053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3720fe-3594-4c27-ac51-07cb487d348b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:16.056721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3720fe-3594-4c27-ac51-07cb487d348b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:16.068271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1079/2192] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:00:16.072505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:16.083153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dabe1-4564-43dc-9962-6a4f54e8d9b6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:16.087807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:16.088749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:16.089034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:16.089311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] PATH_INFO: `/v3/projects/af77fdfaf6cb4ae18d33d3ae4bac9459/users/e929fc58e8e24db8b5ac4f7976b54320/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:16.091336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af77fdfaf6cb4ae18d33d3ae4bac9459, user_id=e929fc58e8e24db8b5ac4f7976b54320, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:16.094766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:16.105909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1114/2193] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:00:16 2022] PUT /identity/v3/projects/af77fdfaf6cb4ae18d33d3ae4bac9459/users/e929fc58e8e24db8b5ac4f7976b54320/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:21.069428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff54fe-9db4-4fdc-8609-59f6410c1f74 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:21.069881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff54fe-9db4-4fdc-8609-59f6410c1f74 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:21.069979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff54fe-9db4-4fdc-8609-59f6410c1f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:21.122000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c9ff54fe-9db4-4fdc-8609-59f6410c1f74 None None] MFA Rules not processed for user `e929fc58e8e24db8b5ac4f7976b54320`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:21.123320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c9ff54fe-9db4-4fdc-8609-59f6410c1f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:21.297176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1080/2194] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:21.311352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:21.314570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:21.438823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cabad-8e8f-49c9-a590-d9da7172cc39 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:21.455050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:21.455791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:21.456075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:21.456378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:21.526915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:21.542015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:21.543772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:21.563977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1115/2195] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:22.927756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cb889-9749-476a-99be-c6c04e302357 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:22.928239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cb889-9749-476a-99be-c6c04e302357 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:22.928536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cb889-9749-476a-99be-c6c04e302357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:22.946749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4d5cb889-9749-476a-99be-c6c04e302357 None None] MFA Rules not processed for user `5c98ae10605b4af1b367a28870919e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:22.947937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4d5cb889-9749-476a-99be-c6c04e302357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:22.955411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1081/2196] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:25.535851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:25.542351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:25.635024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d13b76-613b-4010-83a3-0d70a86fc484 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:25.640229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:25.641344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:25.641635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:25.641912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] PATH_INFO: `/v3/users/f062e303fea7439c9665fb11cf028fdb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:25.643170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f062e303fea7439c9665fb11cf028fdb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:25.645451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:25.711213 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f062e303fea7439c9665fb11cf028fdb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:25.712050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f062e303fea7439c9665fb11cf028fdb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:25.712820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f062e303fea7439c9665fb11cf028fdb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:25.720632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f062e303fea7439c9665fb11cf028fdb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:25.733235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f062e303fea7439c9665fb11cf028fdb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:25.751160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1116/2197] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:00:25 2022] DELETE /identity/v3/users/f062e303fea7439c9665fb11cf028fdb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:25.756772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:25.757282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:25.757363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:25.776941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:25.778138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:25.853020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1082/2198] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:25.861423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:25.862847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:25.926942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fe905-ac62-4f75-8281-40192ebce904 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:25.933746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0b08a-2cc2-4536-a17f-dbb677220235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:25.934450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0b08a-2cc2-4536-a17f-dbb677220235 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:25.934734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0b08a-2cc2-4536-a17f-dbb677220235 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:25.935004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0b08a-2cc2-4536-a17f-dbb677220235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:25.955536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0b08a-2cc2-4536-a17f-dbb677220235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:25.957799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0b08a-2cc2-4536-a17f-dbb677220235 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:25.971212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1117/2199] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:26.325608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:26.327184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:26.346522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f4acd-84bc-44e4-b9c6-f7530e5ae023 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:26.352950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:26.354493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.355401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.355689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] PATH_INFO: `/v3/projects/92e79292402548189e0f8f0d7de557ef` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.357017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e79292402548189e0f8f0d7de557ef)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:26.360535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:26.409632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e79292402548189e0f8f0d7de557ef', 'request_id': 'req-9984779e-85f2-4523-935b-d3713a3dfa40'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:26.415919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e79292402548189e0f8f0d7de557ef', 'request_id': 'req-9984779e-85f2-4523-935b-d3713a3dfa40'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:26.425471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e79292402548189e0f8f0d7de557ef', 'request_id': 'req-9984779e-85f2-4523-935b-d3713a3dfa40'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:26.427090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:26.440013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] The token cache is being invalidate because project 92e79292402548189e0f8f0d7de557ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:00:26.440301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:26.440976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9984779e-85f2-4523-935b-d3713a3dfa40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:00:26.442513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1083/2200] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:00:26 2022] DELETE /identity/v3/projects/92e79292402548189e0f8f0d7de557ef => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:00:26.447683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3051-c560-458a-b714-bf939caab314 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.448003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3051-c560-458a-b714-bf939caab314 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.448280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3051-c560-458a-b714-bf939caab314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.461127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:26.467581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c88d3051-c560-458a-b714-bf939caab314 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:26.470330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c88d3051-c560-458a-b714-bf939caab314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:26.528090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1118/2201] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:26.536519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562b55e-0f1b-44b6-9394-149bb2b5bad5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.536809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562b55e-0f1b-44b6-9394-149bb2b5bad5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.537086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562b55e-0f1b-44b6-9394-149bb2b5bad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.555090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2562b55e-0f1b-44b6-9394-149bb2b5bad5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:26.556283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2562b55e-0f1b-44b6-9394-149bb2b5bad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:26.568002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1084/2202] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:26.571757 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3051-c560-458a-b714-bf939caab314 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:26.582675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3051-c560-458a-b714-bf939caab314 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:26.587463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:26.588056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.588333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.588656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.589383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:00:26.589683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:26.591285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:26.608421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1119/2203] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:00:26.618777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562b55e-0f1b-44b6-9394-149bb2b5bad5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:26.629841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562b55e-0f1b-44b6-9394-149bb2b5bad5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:26.641206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:26.642121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.642393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.642769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.643549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:26.645488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:26.677402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1085/2204] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:00:26 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:00:26.681710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:26.693110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11144dd-2562-4394-bb63-6828f8346a1b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:26.752566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:26.753239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.753519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.753786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.754462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:26.756456 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:26.776116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1120/2205] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:00:26 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:00:26.780250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:26.790545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e1f52-0604-4bc7-8781-5b5e458898a7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:26.795191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:26.795839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.796120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.796458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.797179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:26.798750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:26.809205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1086/2206] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:00:26.813603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:26.823424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d89942-1c3c-47e3-a987-8d17a647a232 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:26.828944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a043ef-5022-4aa9-9e53-48fa7bfa0ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:26.829584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a043ef-5022-4aa9-9e53-48fa7bfa0ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.829860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a043ef-5022-4aa9-9e53-48fa7bfa0ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.830129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a043ef-5022-4aa9-9e53-48fa7bfa0ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.830843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a043ef-5022-4aa9-9e53-48fa7bfa0ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:26.832499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a043ef-5022-4aa9-9e53-48fa7bfa0ae4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:26.846696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:26.846853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1121/2207] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:00:26.857135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589863ae-11a6-4b7f-89a9-2bb1c263a7cb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:26.861925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:26.862790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.863070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.863373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] PATH_INFO: `/v3/projects/885d09051d4e42f98a1a480e0356e0a8/users/674a4f4adb6e4ace8fc4cc0ef7230aa8/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.879954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=885d09051d4e42f98a1a480e0356e0a8, user_id=674a4f4adb6e4ace8fc4cc0ef7230aa8, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:26.882738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:26.898250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1087/2208] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:00:26 2022] PUT /identity/v3/projects/885d09051d4e42f98a1a480e0356e0a8/users/674a4f4adb6e4ace8fc4cc0ef7230aa8/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:00:26.909562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ca538-aa0a-499c-b7cc-7fd020a36bce None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:26.909853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ca538-aa0a-499c-b7cc-7fd020a36bce None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:26.910131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ca538-aa0a-499c-b7cc-7fd020a36bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:26.941502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3f4ca538-aa0a-499c-b7cc-7fd020a36bce None None] MFA Rules not processed for user `674a4f4adb6e4ace8fc4cc0ef7230aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:26.942804 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3f4ca538-aa0a-499c-b7cc-7fd020a36bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:27.010832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1122/2209] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:27.022376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:27.024429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:27.108580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef115df-44b4-484c-aaac-4d0089de85cc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:27.113481 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:27.114075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:27.114363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:27.114644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:27.124836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:27.126380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:27.137111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1088/2210] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:32.073874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc327a2-8fc5-4641-8c39-b060f18526df None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:32.074364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc327a2-8fc5-4641-8c39-b060f18526df None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:32.074452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc327a2-8fc5-4641-8c39-b060f18526df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:32.080064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:32.092877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bcc327a2-8fc5-4641-8c39-b060f18526df None None] MFA Rules not processed for user `c26cd29ec472479a9a36ac9bb1f05580`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:32.094053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bcc327a2-8fc5-4641-8c39-b060f18526df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:32.150504 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1123/2211] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:37.132914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=2,keystone:SELECT=56,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:42.143802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=24 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:00:52.005432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:52.006970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:52.073456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9c4ec-e8c6-4f9d-84de-610c1f18a451 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:52.077083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be17233-f5b7-4369-a36b-144e52538899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:52.077740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be17233-f5b7-4369-a36b-144e52538899 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:52.078021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be17233-f5b7-4369-a36b-144e52538899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:52.078297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be17233-f5b7-4369-a36b-144e52538899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:52.080049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7be17233-f5b7-4369-a36b-144e52538899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:52.164566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be17233-f5b7-4369-a36b-144e52538899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:52.166008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be17233-f5b7-4369-a36b-144e52538899 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:52.175610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1089/2212] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:00:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:00:56.187365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:56.187902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:56.188025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:56.230086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] MFA Rules not processed for user `f47acc995d80476cbd4153e9121e97eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:56.231582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:56.313023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1124/2213] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:56.726756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71474c4b-e96f-4474-89af-0b7de8922633 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:56.727276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71474c4b-e96f-4474-89af-0b7de8922633 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:56.727372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71474c4b-e96f-4474-89af-0b7de8922633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:56.746896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-71474c4b-e96f-4474-89af-0b7de8922633 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:00:56.748423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-71474c4b-e96f-4474-89af-0b7de8922633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:56.827176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1090/2214] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:00:56.879696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:00:56.881336 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:00:56.966698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daf2c6-4dfa-4b3e-878b-7fa7bd7e6dae None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:00:56.972777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74b9cd-75f0-478d-89ad-cc59e625af0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:00:56.974028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74b9cd-75f0-478d-89ad-cc59e625af0f service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:00:56.974311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74b9cd-75f0-478d-89ad-cc59e625af0f service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:00:56.974585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74b9cd-75f0-478d-89ad-cc59e625af0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:00:56.983832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74b9cd-75f0-478d-89ad-cc59e625af0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:00:56.985714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74b9cd-75f0-478d-89ad-cc59e625af0f service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:00:56.995951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1125/2215] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:00:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:01:06.820576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=82 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:06.992093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=72 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:07.191984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71474c4b-e96f-4474-89af-0b7de8922633 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:07.350636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71474c4b-e96f-4474-89af-0b7de8922633 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:07.355969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:07.356871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:07.357261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:07.358212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] PATH_INFO: `/v3/users/674a4f4adb6e4ace8fc4cc0ef7230aa8` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:07.359052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=674a4f4adb6e4ace8fc4cc0ef7230aa8)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:07.361015 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:07.421750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674a4f4adb6e4ace8fc4cc0ef7230aa8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:07.423614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674a4f4adb6e4ace8fc4cc0ef7230aa8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:07.440313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674a4f4adb6e4ace8fc4cc0ef7230aa8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:07.441379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674a4f4adb6e4ace8fc4cc0ef7230aa8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:07.454984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674a4f4adb6e4ace8fc4cc0ef7230aa8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:07.478371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1091/2216] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:01:07 2022] DELETE /identity/v3/users/674a4f4adb6e4ace8fc4cc0ef7230aa8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:01:07.483153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:07.483572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:07.483945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:07.533405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:07.537153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:07.621077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1126/2217] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:07.634995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:07.637161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:07.745784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:07.758870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5859a10-4257-4ccb-8c45-9228e0b672a3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:07.766865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab8eef-cda7-4c84-b786-d77a145ec3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:07.767402 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab8eef-cda7-4c84-b786-d77a145ec3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:07.767798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab8eef-cda7-4c84-b786-d77a145ec3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:07.768557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab8eef-cda7-4c84-b786-d77a145ec3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:07.783794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab8eef-cda7-4c84-b786-d77a145ec3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:07.785826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab8eef-cda7-4c84-b786-d77a145ec3d8 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:07.808436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1092/2218] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:01:08.344155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:08.345677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:08.358439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c109e0b-15ee-4455-93a7-f8df47c48435 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:08.364173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:08.365450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.365733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.366010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] PATH_INFO: `/v3/projects/885d09051d4e42f98a1a480e0356e0a8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.367226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=885d09051d4e42f98a1a480e0356e0a8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:08.369188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:08.431257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885d09051d4e42f98a1a480e0356e0a8', 'request_id': 'req-eb74a46f-b306-4579-a7a5-b59215a56c6a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:08.437524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885d09051d4e42f98a1a480e0356e0a8', 'request_id': 'req-eb74a46f-b306-4579-a7a5-b59215a56c6a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:08.444588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885d09051d4e42f98a1a480e0356e0a8', 'request_id': 'req-eb74a46f-b306-4579-a7a5-b59215a56c6a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:08.457826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] The token cache is being invalidate because project 885d09051d4e42f98a1a480e0356e0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:01:08.458108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:08.458715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-eb74a46f-b306-4579-a7a5-b59215a56c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:08.460253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1127/2219] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:01:08 2022] DELETE /identity/v3/projects/885d09051d4e42f98a1a480e0356e0a8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:01:08.465076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999cb1e-f832-4fba-a17a-598da51a07ad None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.465586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999cb1e-f832-4fba-a17a-598da51a07ad None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.465676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999cb1e-f832-4fba-a17a-598da51a07ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.515743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d999cb1e-f832-4fba-a17a-598da51a07ad None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:08.516999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d999cb1e-f832-4fba-a17a-598da51a07ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:08.603180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1093/2220] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:08.610685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb00be-9da5-4e00-9e4d-f4ead5380364 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.610967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb00be-9da5-4e00-9e4d-f4ead5380364 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.611240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb00be-9da5-4e00-9e4d-f4ead5380364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.630716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5efb00be-9da5-4e00-9e4d-f4ead5380364 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:08.632547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5efb00be-9da5-4e00-9e4d-f4ead5380364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:08.642169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1128/2221] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:08.647130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999cb1e-f832-4fba-a17a-598da51a07ad None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:08.657521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999cb1e-f832-4fba-a17a-598da51a07ad None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:08.662511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:08.663176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.663455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.663729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.664491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:01:08.664797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:08.666479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:08.679430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1094/2222] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:01:08.683369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb00be-9da5-4e00-9e4d-f4ead5380364 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:08.693924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb00be-9da5-4e00-9e4d-f4ead5380364 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:08.698663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:08.699351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.699635 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.699909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.700642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:08.702521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:08.735415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1129/2223] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:01:08 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:01:08.736555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:08.747049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac84d-2edd-4b09-b801-a5cfee1af780 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:08.809581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:08.810258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.810541 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.810819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.811522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:08.813517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:08.838774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1095/2224] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:01:08 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:01:08.840014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:08.850778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c442cb3-c660-405b-9a18-ad06d74d7d3b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:08.855714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:08.856395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.856683 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.856959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.857682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:08.859266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:08.870104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1130/2225] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:01:08.873873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:08.884816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342a3ef-fc7c-4721-9023-7a6c1a4e72c0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:08.889466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1366b-0b89-43b7-a9f6-e0dec7d40d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:08.890400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1366b-0b89-43b7-a9f6-e0dec7d40d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.890806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1366b-0b89-43b7-a9f6-e0dec7d40d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.891389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1366b-0b89-43b7-a9f6-e0dec7d40d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.891802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1366b-0b89-43b7-a9f6-e0dec7d40d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:08.893441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1366b-0b89-43b7-a9f6-e0dec7d40d90 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:08.904489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1096/2226] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:01:08.908561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:08.918939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c71b67-af47-45ca-a91b-7a64b4d9f103 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:08.923642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:08.924554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.924841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.925116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] PATH_INFO: `/v3/projects/4bd3c10c39cd4e2282cc954487fee86c/users/abbcee9af1d9400ba851d2b8b24e6532/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.941947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd3c10c39cd4e2282cc954487fee86c, user_id=abbcee9af1d9400ba851d2b8b24e6532, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:08.944815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:08.958894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1131/2227] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:01:08 2022] PUT /identity/v3/projects/4bd3c10c39cd4e2282cc954487fee86c/users/abbcee9af1d9400ba851d2b8b24e6532/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:01:08.969136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:08.969416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:08.969689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:08.998834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] MFA Rules not processed for user `abbcee9af1d9400ba851d2b8b24e6532`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:09.000083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:09.014831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:09.070937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1097/2228] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:09.082104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:09.083877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:09.089505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:09.152507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f592d1-7d1f-4fb8-86ed-316b56946394 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:09.155727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acefc97-33ff-4441-904b-e7a450aeb090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:09.156388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acefc97-33ff-4441-904b-e7a450aeb090 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:09.156671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acefc97-33ff-4441-904b-e7a450aeb090 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:09.156937 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acefc97-33ff-4441-904b-e7a450aeb090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:09.165633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acefc97-33ff-4441-904b-e7a450aeb090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:09.167066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acefc97-33ff-4441-904b-e7a450aeb090 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:09.176308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1132/2229] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:01:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:01:09.806605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:09.808076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:09.867439 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed413c86-0eda-4332-a6fb-398916b73ae4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:09.871062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92eca-faee-407a-8e6d-1e8a324c1655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:09.871713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92eca-faee-407a-8e6d-1e8a324c1655 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:09.872067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92eca-faee-407a-8e6d-1e8a324c1655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:09.872459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92eca-faee-407a-8e6d-1e8a324c1655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:09.874322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1de92eca-faee-407a-8e6d-1e8a324c1655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:09.942459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de92eca-faee-407a-8e6d-1e8a324c1655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:09.943834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de92eca-faee-407a-8e6d-1e8a324c1655 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:09.953565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1098/2230] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:01:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:01:10.565798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a4a4f-8857-487f-aa9c-ab9826a1c71b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:10.566927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a4a4f-8857-487f-aa9c-ab9826a1c71b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:10.567102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a4a4f-8857-487f-aa9c-ab9826a1c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:10.584573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5d0a4a4f-8857-487f-aa9c-ab9826a1c71b None None] MFA Rules not processed for user `abbcee9af1d9400ba851d2b8b24e6532`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:10.585767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5d0a4a4f-8857-487f-aa9c-ab9826a1c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:10.592633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1133/2231] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:01:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:14.564374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c32f04-22b4-42b3-805b-bf875fa846e0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:14.564899 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c32f04-22b4-42b3-805b-bf875fa846e0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:14.564983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c32f04-22b4-42b3-805b-bf875fa846e0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:14.566579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1099/2232] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 16:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 16:01:14.581517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882732d1-0744-40cd-91c9-d7938a0b7231 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:14.582220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882732d1-0744-40cd-91c9-d7938a0b7231 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:14.582220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882732d1-0744-40cd-91c9-d7938a0b7231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:14.617157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-882732d1-0744-40cd-91c9-d7938a0b7231 None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:14.618313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-882732d1-0744-40cd-91c9-d7938a0b7231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:14.702539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1134/2233] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 16:01:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:01:14.719355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c32f04-22b4-42b3-805b-bf875fa846e0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:14.721283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-65c32f04-22b4-42b3-805b-bf875fa846e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:14.799924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c32f04-22b4-42b3-805b-bf875fa846e0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:14.804742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb28088-234f-4e20-95fc-7e9f01c3f786 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0206e09e4f426d9f4b54543d89c039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:14.805403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28088-234f-4e20-95fc-7e9f01c3f786 service glance] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:14.805686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28088-234f-4e20-95fc-7e9f01c3f786 service glance] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:14.805962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28088-234f-4e20-95fc-7e9f01c3f786 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:14.818004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb28088-234f-4e20-95fc-7e9f01c3f786 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:14.819442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb28088-234f-4e20-95fc-7e9f01c3f786 service glance] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:14.831122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1100/2234] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:01:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:01:14.911061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e3790-8aef-4c84-8803-bebe6ba4d6d3 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:14.911061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e3790-8aef-4c84-8803-bebe6ba4d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:14.911061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e3790-8aef-4c84-8803-bebe6ba4d6d3 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:14.911061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1135/2235] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 16:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 16:01:14.912693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b68d7-b0c0-4e65-b145-30791587adb1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:14.913033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b68d7-b0c0-4e65-b145-30791587adb1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:14.913321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b68d7-b0c0-4e65-b145-30791587adb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:14.934203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e61b68d7-b0c0-4e65-b145-30791587adb1 None None] MFA Rules not processed for user `dba9f1fa257a4f44ae159850f9ba349c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:14.935609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e61b68d7-b0c0-4e65-b145-30791587adb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:14.943895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1101/2236] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 16:01:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:14.956123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e3790-8aef-4c84-8803-bebe6ba4d6d3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:14.970667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e3790-8aef-4c84-8803-bebe6ba4d6d3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:14.975007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c16c9-d40e-4a9f-aa6c-f9e63c2d1c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:14.975667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c16c9-d40e-4a9f-aa6c-f9e63c2d1c20 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:14.975949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c16c9-d40e-4a9f-aa6c-f9e63c2d1c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:14.976311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c16c9-d40e-4a9f-aa6c-f9e63c2d1c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:14.985347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c16c9-d40e-4a9f-aa6c-f9e63c2d1c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:14.986973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c16c9-d40e-4a9f-aa6c-f9e63c2d1c20 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:14.996555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1136/2237] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:01:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:01:15.085281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f599-c3b9-4767-96bb-05fbd6c4d221 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:15.085566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f599-c3b9-4767-96bb-05fbd6c4d221 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:15.086555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f599-c3b9-4767-96bb-05fbd6c4d221 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:15.089023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1102/2238] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 16:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 16:01:15.096289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce15a2-7e98-4de0-861d-85c54750626f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:15.096922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce15a2-7e98-4de0-861d-85c54750626f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:15.096922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce15a2-7e98-4de0-861d-85c54750626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:15.121620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ddce15a2-7e98-4de0-861d-85c54750626f None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:15.122879 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ddce15a2-7e98-4de0-861d-85c54750626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:15.139749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1137/2239] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:15.151275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9f599-c3b9-4767-96bb-05fbd6c4d221 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:15.157572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:15.166421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9f599-c3b9-4767-96bb-05fbd6c4d221 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:15.171148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:15.172157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:15.172624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:15.173074 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:15.186869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:15.188380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:15.203793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1103/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:01:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:01:19.290209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237eef92-fc26-4a1a-bd0e-1eb7d982efd3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:19.290699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237eef92-fc26-4a1a-bd0e-1eb7d982efd3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:19.290919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237eef92-fc26-4a1a-bd0e-1eb7d982efd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:19.309726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-237eef92-fc26-4a1a-bd0e-1eb7d982efd3 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:19.311061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-237eef92-fc26-4a1a-bd0e-1eb7d982efd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:19.387484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:19.396271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1138/2241] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:01:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:19.408290 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:19.409813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:19.479080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f555da-ff5a-4f62-8822-023a2c1a24c2 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:19.484202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:19.484877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:19.485184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:19.485460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:19.494035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:19.495618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:19.503582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1104/2242] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:01:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:01:29.500448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=37 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:39.961763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f3fee-91a0-44ed-a15e-815d52d86046 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:39.962276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f3fee-91a0-44ed-a15e-815d52d86046 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:39.962369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f3fee-91a0-44ed-a15e-815d52d86046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:39.981960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a69f3fee-91a0-44ed-a15e-815d52d86046 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:39.983183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a69f3fee-91a0-44ed-a15e-815d52d86046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:39.990860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1139/2243] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:40.002547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:40.013241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e1d7-a57b-4689-8ab2-441b268e07f5 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:40.018421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:40.019054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:40.019388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:40.019833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:40.028099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:40.029711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:40.039113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1105/2244] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:01:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:01:40.580591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4693a23-8cbc-4037-87d4-757ff493ef91 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:40.580884 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4693a23-8cbc-4037-87d4-757ff493ef91 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:40.581165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4693a23-8cbc-4037-87d4-757ff493ef91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:40.598762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c4693a23-8cbc-4037-87d4-757ff493ef91 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:40.600072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c4693a23-8cbc-4037-87d4-757ff493ef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:40.607417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1140/2245] 10.222.0.18 () {58 vars in 967 bytes} [Tue Jun 7 16:01:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:40.620177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:40.631178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7858ed-9756-4137-a55c-c4d36213b90a service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:40.636177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:40.636831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:40.637109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:40.637379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:40.646211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:40.647785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:40.656210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1106/2246] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:01:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:01:40.897488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7b521-ef46-43ec-91b4-e13e03269066 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:40.897777 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7b521-ef46-43ec-91b4-e13e03269066 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:40.898051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7b521-ef46-43ec-91b4-e13e03269066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:40.916324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6ca7b521-ef46-43ec-91b4-e13e03269066 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:40.917555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6ca7b521-ef46-43ec-91b4-e13e03269066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:40.925320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1141/2247] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:01:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:40.938899 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:40.949994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec2883-9de8-4a85-8f3c-5fec319d600e service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:40.954967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:40.955600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:40.955879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:40.956149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:40.964261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:40.965860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:40.974226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1107/2248] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:01:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:01:41.297278 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c42422-1283-4406-ba76-d90227dbae22 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:41.297736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c42422-1283-4406-ba76-d90227dbae22 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:41.297849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c42422-1283-4406-ba76-d90227dbae22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:41.315955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a1c42422-1283-4406-ba76-d90227dbae22 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:41.317767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a1c42422-1283-4406-ba76-d90227dbae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:41.324932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1142/2249] 10.222.0.18 () {58 vars in 967 bytes} [Tue Jun 7 16:01:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:41.337284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:41.347752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62722a33-6af4-43f3-8aa2-1bd29a3d6f0b service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:41.352824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c74921-2923-443b-9616-01fadddbbacd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:41.353473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c74921-2923-443b-9616-01fadddbbacd service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:41.353757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c74921-2923-443b-9616-01fadddbbacd service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:41.354156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c74921-2923-443b-9616-01fadddbbacd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:41.362642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c74921-2923-443b-9616-01fadddbbacd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:41.364829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c74921-2923-443b-9616-01fadddbbacd service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:41.373014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1108/2250] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:01:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:01:43.496393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c42422-1283-4406-ba76-d90227dbae22 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:43.559618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c42422-1283-4406-ba76-d90227dbae22 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:43.564033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:43.564935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:43.565212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:43.565483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] PATH_INFO: `/v3/users/abbcee9af1d9400ba851d2b8b24e6532` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:43.566650 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbcee9af1d9400ba851d2b8b24e6532)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:43.568780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:43.615486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbcee9af1d9400ba851d2b8b24e6532'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:43.616126 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbcee9af1d9400ba851d2b8b24e6532'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:43.616742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbcee9af1d9400ba851d2b8b24e6532'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:43.623522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbcee9af1d9400ba851d2b8b24e6532'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:43.635086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbcee9af1d9400ba851d2b8b24e6532'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:43.647374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1143/2251] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:01:43 2022] DELETE /identity/v3/users/abbcee9af1d9400ba851d2b8b24e6532 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:01:43.651972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:43.652485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:43.652687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:43.670507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:43.671697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:43.729550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1109/2252] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:43.739669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:43.741076 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:43.797805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b1182-d965-4096-b068-5dcbf0c2ef2b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:43.803134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f3777-3619-4089-a0cf-fd5e73ca4710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:43.803862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f3777-3619-4089-a0cf-fd5e73ca4710 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:43.804190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f3777-3619-4089-a0cf-fd5e73ca4710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:43.804514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f3777-3619-4089-a0cf-fd5e73ca4710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:43.811898 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:43.814546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f3777-3619-4089-a0cf-fd5e73ca4710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:43.817541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f3777-3619-4089-a0cf-fd5e73ca4710 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:43.828780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1144/2253] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:01:44.000181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:44.001740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:44.018858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f4b63-61c6-4d47-904d-626218ef788c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:44.023859 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:44.024751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:44.025187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:44.025297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] PATH_INFO: `/v3/projects/4bd3c10c39cd4e2282cc954487fee86c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:44.026471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd3c10c39cd4e2282cc954487fee86c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:44.028968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:44.081039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd3c10c39cd4e2282cc954487fee86c', 'request_id': 'req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:44.088215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd3c10c39cd4e2282cc954487fee86c', 'request_id': 'req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:44.097331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd3c10c39cd4e2282cc954487fee86c', 'request_id': 'req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:44.110171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] The token cache is being invalidate because project 4bd3c10c39cd4e2282cc954487fee86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:01:44.110448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:44.111104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:01:44.112684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1110/2254] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:01:43 2022] DELETE /identity/v3/projects/4bd3c10c39cd4e2282cc954487fee86c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:01:45.529323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9163fb-f05b-41e3-bc29-a569281622ff None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:45.529812 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9163fb-f05b-41e3-bc29-a569281622ff None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:45.530029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9163fb-f05b-41e3-bc29-a569281622ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:45.580844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ea9163fb-f05b-41e3-bc29-a569281622ff None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:01:45.582060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ea9163fb-f05b-41e3-bc29-a569281622ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:45.651302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1145/2255] 10.222.0.18 () {58 vars in 967 bytes} [Tue Jun 7 16:01:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:01:45.664177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:01:45.665597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:01:45.698664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:45.757792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4a097-77fd-4590-ade8-a9eed7bb60dc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:01:45.763396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:01:45.764045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:01:45.764324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:01:45.764630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:01:45.774416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:01:45.775997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:01:45.783918 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1111/2256] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:01:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:01:55.648465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=42 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:01:55.780721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=27 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:02:08.464548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457f5c-d31b-4f01-8685-9330a46c5e99 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:02:08.465075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457f5c-d31b-4f01-8685-9330a46c5e99 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:02:08.465186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457f5c-d31b-4f01-8685-9330a46c5e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:02:08.483594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2c457f5c-d31b-4f01-8685-9330a46c5e99 None None] MFA Rules not processed for user `e9ca987fc23d4ff1a88dd4ed7b38980c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:02:08.484735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2c457f5c-d31b-4f01-8685-9330a46c5e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:02:08.540275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1146/2257] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:02:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:02:09.594473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:02:09.595896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:02:09.690233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e31f-561d-414e-96ae-029094bc0df1 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:02:09.696819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbab41-0488-4b30-8bbc-0cbc176f7ac9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:02:09.697471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbab41-0488-4b30-8bbc-0cbc176f7ac9 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:02:09.697756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbab41-0488-4b30-8bbc-0cbc176f7ac9 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:02:09.698034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbab41-0488-4b30-8bbc-0cbc176f7ac9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:02:09.699792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-72fbab41-0488-4b30-8bbc-0cbc176f7ac9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:02:09.712974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbab41-0488-4b30-8bbc-0cbc176f7ac9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:02:09.714573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbab41-0488-4b30-8bbc-0cbc176f7ac9 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:02:09.724611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1112/2258] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:02:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:02:18.532975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=25 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:02:19.720313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=41 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:02:26.434719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec25735-2415-47c4-a181-e5e0d5ce7a99 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:02:26.435192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec25735-2415-47c4-a181-e5e0d5ce7a99 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:02:26.435298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec25735-2415-47c4-a181-e5e0d5ce7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:02:26.455013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-eec25735-2415-47c4-a181-e5e0d5ce7a99 None None] MFA Rules not processed for user `f47acc995d80476cbd4153e9121e97eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:02:26.456318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eec25735-2415-47c4-a181-e5e0d5ce7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:02:26.515032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1147/2259] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:02:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:02:36.508238 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=25 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:02:37.441578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21a37-9773-4373-ba61-e91527033c17 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:02:37.442274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21a37-9773-4373-ba61-e91527033c17 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:02:37.442274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21a37-9773-4373-ba61-e91527033c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:02:37.463336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9ba21a37-9773-4373-ba61-e91527033c17 None None] MFA Rules not processed for user `213a7f6737504f9eb89a2e51ddd639ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:02:37.464557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9ba21a37-9773-4373-ba61-e91527033c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:02:37.528610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1113/2260] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:02:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:02:47.521185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=25 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:03:09.431772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680f6db-0192-4d83-8bb9-10147710df60 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:09.432287 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680f6db-0192-4d83-8bb9-10147710df60 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:09.432411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680f6db-0192-4d83-8bb9-10147710df60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:09.449874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d680f6db-0192-4d83-8bb9-10147710df60 None None] MFA Rules not processed for user `8348af929ea740f3981631020a219f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:03:09.451087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d680f6db-0192-4d83-8bb9-10147710df60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:09.508704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1148/2261] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:03:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:03:19.499999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=25 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:03:19.760606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8d18-5035-4192-9d8b-2629f1c7e58d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:19.761099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8d18-5035-4192-9d8b-2629f1c7e58d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:19.761207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8d18-5035-4192-9d8b-2629f1c7e58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:19.781764 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5cce8d18-5035-4192-9d8b-2629f1c7e58d None None] MFA Rules not processed for user `c26cd29ec472479a9a36ac9bb1f05580`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:03:19.783001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5cce8d18-5035-4192-9d8b-2629f1c7e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:19.839963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1114/2262] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:03:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:03:29.833021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=25 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:03:43.395676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:43.398308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:43.398806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:43.419964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] MFA Rules not processed for user `f47acc995d80476cbd4153e9121e97eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:03:43.421802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:43.436821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1149/2263] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:03:44.139198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e59bf-889c-4632-981d-1fb8138a947e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:44.139758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e59bf-889c-4632-981d-1fb8138a947e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:44.139758 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e59bf-889c-4632-981d-1fb8138a947e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:44.161184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-865e59bf-889c-4632-981d-1fb8138a947e None None] MFA Rules not processed for user `e9ca987fc23d4ff1a88dd4ed7b38980c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:03:44.163163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-865e59bf-889c-4632-981d-1fb8138a947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:44.170790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1115/2264] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:03:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:03:44.338801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:03:44.341269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:44.415364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaf540-a1fc-41a1-b6a1-6973a7be12f6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:03:44.419745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:03:44.420666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:44.421046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:44.421487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:44.424151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:44.557761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:03:44.561209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:03:44.588984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1150/2265] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:03:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:03:46.451490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e59bf-889c-4632-981d-1fb8138a947e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:03:46.462558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e59bf-889c-4632-981d-1fb8138a947e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:03:46.466653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d693b-2ff5-4bb3-8542-906015514f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:03:46.467213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d693b-2ff5-4bb3-8542-906015514f0e service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:46.467498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d693b-2ff5-4bb3-8542-906015514f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:46.467774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d693b-2ff5-4bb3-8542-906015514f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:46.470197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7a5d693b-2ff5-4bb3-8542-906015514f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:46.486018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d693b-2ff5-4bb3-8542-906015514f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:03:46.487574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d693b-2ff5-4bb3-8542-906015514f0e service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:03:46.500316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1116/2266] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:03:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:03:46.908258 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:03:46.917658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e7e96-ab76-4067-b7b8-7bfcc7b08514 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:03:46.921334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feac85-1360-4d75-955a-b2aed788dc9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:03:46.921975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feac85-1360-4d75-955a-b2aed788dc9d service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:46.922262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feac85-1360-4d75-955a-b2aed788dc9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:46.922539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feac85-1360-4d75-955a-b2aed788dc9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:46.924505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f2feac85-1360-4d75-955a-b2aed788dc9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:46.934719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feac85-1360-4d75-955a-b2aed788dc9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:03:46.936131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feac85-1360-4d75-955a-b2aed788dc9d service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:03:46.944830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1151/2267] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:03:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:03:47.074617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ddc5-5bcf-4017-a5cb-570a529d8429 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:47.075163 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ddc5-5bcf-4017-a5cb-570a529d8429 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:47.075607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ddc5-5bcf-4017-a5cb-570a529d8429 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:47.077174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1117/2268] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 16:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 16:03:47.083585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c692c-c662-4394-be05-889f4bfa6ada None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:47.083918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c692c-c662-4394-be05-889f4bfa6ada None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:47.084202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c692c-c662-4394-be05-889f4bfa6ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:47.105216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-5e2c692c-c662-4394-be05-889f4bfa6ada None None] MFA Rules not processed for user `dba9f1fa257a4f44ae159850f9ba349c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:03:47.106679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5e2c692c-c662-4394-be05-889f4bfa6ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:47.113489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1152/2269] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 16:03:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:03:47.126488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ddc5-5bcf-4017-a5cb-570a529d8429 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:03:47.128658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e312ddc5-5bcf-4017-a5cb-570a529d8429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:47.142674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ddc5-5bcf-4017-a5cb-570a529d8429 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:03:47.150965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a595681-1dc8-4ab5-9cd6-bfae2d22915a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:03:47.151622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a595681-1dc8-4ab5-9cd6-bfae2d22915a service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:47.152016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a595681-1dc8-4ab5-9cd6-bfae2d22915a service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:47.152389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a595681-1dc8-4ab5-9cd6-bfae2d22915a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:47.161205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a595681-1dc8-4ab5-9cd6-bfae2d22915a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:03:47.162851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a595681-1dc8-4ab5-9cd6-bfae2d22915a service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:03:47.182888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1118/2270] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:03:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:03:47.290839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62a3cb-d217-428e-a89a-cc72a91336f5 None None] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:47.291232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62a3cb-d217-428e-a89a-cc72a91336f5 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:47.291384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62a3cb-d217-428e-a89a-cc72a91336f5 None None] PATH_INFO: `/` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:47.293091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1153/2271] 10.222.0.18 () {54 vars in 921 bytes} [Tue Jun 7 16:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 16:03:47.299240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108aa6c-e4a3-424a-9b07-4187fc806e20 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:47.299544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108aa6c-e4a3-424a-9b07-4187fc806e20 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:47.299799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108aa6c-e4a3-424a-9b07-4187fc806e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:47.347742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3108aa6c-e4a3-424a-9b07-4187fc806e20 None None] MFA Rules not processed for user `0b352e6a51fc4d1f8b9b97d092d11bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:03:47.349038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3108aa6c-e4a3-424a-9b07-4187fc806e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:47.359565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1119/2272] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Jun 7 16:03:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:03:47.371368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62a3cb-d217-428e-a89a-cc72a91336f5 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:03:47.382893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62a3cb-d217-428e-a89a-cc72a91336f5 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:03:47.386831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e83d-6169-4d2b-826e-5f3abf714db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:03:47.387599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e83d-6169-4d2b-826e-5f3abf714db9 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:47.387927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e83d-6169-4d2b-826e-5f3abf714db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:47.388202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e83d-6169-4d2b-826e-5f3abf714db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:47.399619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e83d-6169-4d2b-826e-5f3abf714db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:03:47.401186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e83d-6169-4d2b-826e-5f3abf714db9 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:03:47.413567 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1154/2273] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:03:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:03:47.483077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108aa6c-e4a3-424a-9b07-4187fc806e20 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:03:47.505368 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108aa6c-e4a3-424a-9b07-4187fc806e20 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:03:47.512070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:03:47.513110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:47.513490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:47.513866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:47.515787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:47.531042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:03:47.533241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:03:47.546480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1120/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:03:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:03:57.407547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=81 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:03:57.541196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=30 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:03:57.832373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62140595-de65-4c44-b71c-d3dc3fcd5cec None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:57.832956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62140595-de65-4c44-b71c-d3dc3fcd5cec None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:57.833445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62140595-de65-4c44-b71c-d3dc3fcd5cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:57.851802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-62140595-de65-4c44-b71c-d3dc3fcd5cec None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:03:57.853445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-62140595-de65-4c44-b71c-d3dc3fcd5cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:03:57.861519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1155/2275] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:03:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:03:57.909336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:03:57.921427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce625bc3-1208-4207-a4fe-36511ec28382 service placement] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:03:57.926389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:03:57.927022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:03:57.927921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:03:57.928425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:03:57.936815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:03:57.938390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:03:57.946483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1121/2276] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:03:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:04:00.359477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62140595-de65-4c44-b71c-d3dc3fcd5cec None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:00.373557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62140595-de65-4c44-b71c-d3dc3fcd5cec None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:00.384769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:00.390631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:00.391977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:00.392499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:00.395738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:00.415036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:00.416938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:00.431139 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1156/2277] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:07.943689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=6 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:07.964299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:07.965667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:07.981257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c7804-8d06-426c-995b-c2a3d171ea40 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:07.986855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab9425-44a2-4942-af2b-0558785f6f66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:07.987490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab9425-44a2-4942-af2b-0558785f6f66 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:07.987772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab9425-44a2-4942-af2b-0558785f6f66 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:07.988159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab9425-44a2-4942-af2b-0558785f6f66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:07.990431 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b1ab9425-44a2-4942-af2b-0558785f6f66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:08.001414 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab9425-44a2-4942-af2b-0558785f6f66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:08.002988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab9425-44a2-4942-af2b-0558785f6f66 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:08.013633 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1122/2278] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:09.133180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:09.146799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02c28d-a9cb-485d-859a-ec989212d0d4 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:09.152669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:09.153316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:09.153624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:09.153900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:09.155703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:09.166631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:09.168212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:09.177313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1157/2279] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:13.165790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090e9eb-ac62-4cf5-b4f8-4f9a6ed925f2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:13.166213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090e9eb-ac62-4cf5-b4f8-4f9a6ed925f2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:13.166332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090e9eb-ac62-4cf5-b4f8-4f9a6ed925f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:13.184216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4090e9eb-ac62-4cf5-b4f8-4f9a6ed925f2 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:04:13.185413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4090e9eb-ac62-4cf5-b4f8-4f9a6ed925f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:13.192315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1123/2280] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:04:13.203813 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:13.215572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d8b1-9890-4e16-a360-b839ab278bbc service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:13.220598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a619e8-a670-4f95-89f1-af9048964b99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:13.221224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a619e8-a670-4f95-89f1-af9048964b99 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:13.221508 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a619e8-a670-4f95-89f1-af9048964b99 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:13.221786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a619e8-a670-4f95-89f1-af9048964b99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:13.230370 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a619e8-a670-4f95-89f1-af9048964b99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:13.232197 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a619e8-a670-4f95-89f1-af9048964b99 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:13.240152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1158/2281] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:04:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 16:04:23.179441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=9 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:23.237017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=21 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:27.757280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090e9eb-ac62-4cf5-b4f8-4f9a6ed925f2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:27.772078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090e9eb-ac62-4cf5-b4f8-4f9a6ed925f2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:27.778228 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:27.778895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:27.779181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:27.779461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:27.781259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:27.794033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:27.795620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:27.806199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1124/2282] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:29.474619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:29.475093 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:29.475192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:29.493193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:04:29.494390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:29.501786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1159/2283] 10.222.0.18 () {58 vars in 967 bytes} [Tue Jun 7 16:04:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:04:29.514170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:29.525195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7829-0999-49f5-bcc8-728fe100b300 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:29.530865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:29.531497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:29.531780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:29.532056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:29.540309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:29.542080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:29.550241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1125/2284] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:04:39.488055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:39.546963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=12 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:40.966769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:40.968224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:40.986563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee4d67-ba09-4bd3-9693-08660b1a86db None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:40.995380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:40.996995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:40.997477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:40.998097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:41.000564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:41.013497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:41.015119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:41.027347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1160/2285] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:04:47.633420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:47.647770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6477cad-ce23-49a2-89de-3d083edaac87 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:47.656889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:47.657400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:47.657683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:47.658108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:47.659766 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:47.670625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:47.672330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:47.682276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1126/2286] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:50.003806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:50.013132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc60103-091f-48f3-a2b2-19a42e27d0ad service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:50.017754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:50.018381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:50.018644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:50.018913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:50.020914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:50.088105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:50.089589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:50.100314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1161/2287] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:53.343246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:53.345820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:53.368267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89c147-c6df-4e80-ae79-7a7733051de3 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:53.375125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:53.376412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:53.376909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:53.377348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:53.379518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:53.398781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:53.408038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:53.408038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:53.409820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:53.424079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10871812-db57-4eb2-9901-8fd5e5f2901c service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:53.424400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1127/2288] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:53.429373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:53.430398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:53.431863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:53.432148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] PATH_INFO: `/v3/users/8348af929ea740f3981631020a219f80` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:53.433593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8348af929ea740f3981631020a219f80)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:53.437403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:53.491241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8348af929ea740f3981631020a219f80'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.491863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8348af929ea740f3981631020a219f80'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.495086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8348af929ea740f3981631020a219f80'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.505180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8348af929ea740f3981631020a219f80'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.518060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-a3d7e12a-d687-4b08-a005-1017377e19d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8348af929ea740f3981631020a219f80'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.531233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1162/2289] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:04:53 2022] DELETE /identity/v3/users/8348af929ea740f3981631020a219f80 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:04:53.713891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:53.726297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf9ae6-fa07-4ca7-b46d-ebff99d07df6 service placement] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:53.784195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:53.785092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:53.785373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:53.785646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] PATH_INFO: `/v3/projects/9d0733c937754d1182c68bd4ea7f283c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:53.786823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0733c937754d1182c68bd4ea7f283c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:53.788737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:53.833535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0733c937754d1182c68bd4ea7f283c', 'request_id': 'req-b4979970-4277-4cd8-8427-09db946898c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.839378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0733c937754d1182c68bd4ea7f283c', 'request_id': 'req-b4979970-4277-4cd8-8427-09db946898c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.848072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0733c937754d1182c68bd4ea7f283c', 'request_id': 'req-b4979970-4277-4cd8-8427-09db946898c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.858264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] The token cache is being invalidate because project 9d0733c937754d1182c68bd4ea7f283c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:04:53.858542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.859154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b4979970-4277-4cd8-8427-09db946898c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:04:53.860701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1128/2290] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:04:53 2022] DELETE /identity/v3/projects/9d0733c937754d1182c68bd4ea7f283c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:04:53.865624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3614b5-6e7a-4011-868f-8028516190c1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:53.865915 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3614b5-6e7a-4011-868f-8028516190c1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:53.866198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3614b5-6e7a-4011-868f-8028516190c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:53.885170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6c3614b5-6e7a-4011-868f-8028516190c1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:04:53.886345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6c3614b5-6e7a-4011-868f-8028516190c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:53.931616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:53.944663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1163/2291] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:04:53.953472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfe8b3-22c7-4046-b0f8-3a06dffe4621 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:53.953855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfe8b3-22c7-4046-b0f8-3a06dffe4621 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:53.954146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfe8b3-22c7-4046-b0f8-3a06dffe4621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:53.972199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-78cfe8b3-22c7-4046-b0f8-3a06dffe4621 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:04:53.973409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-78cfe8b3-22c7-4046-b0f8-3a06dffe4621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:53.981131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1129/2292] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:04:53.986120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3614b5-6e7a-4011-868f-8028516190c1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:53.995806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3614b5-6e7a-4011-868f-8028516190c1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:54.009989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:54.010742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.011028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.011313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.012075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:04:54.012404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:54.014028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:54.028271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1164/2293] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:04:54.032803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfe8b3-22c7-4046-b0f8-3a06dffe4621 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:54.042187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfe8b3-22c7-4046-b0f8-3a06dffe4621 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:54.047707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:54.048379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.048676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.048953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.049649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:54.051574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:54.077065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1130/2294] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:04:54 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:04:54.087161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:54.099401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa92ac-8bd2-4b84-92eb-977c9e3ad338 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:54.164453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:54.165136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.165421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.165698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.166396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:54.168335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:54.189607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1165/2295] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:04:54 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:04:54.194045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:54.206445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec60c3-afa3-4280-ab49-692a9137a6f9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:54.211084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:54.211734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.212013 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.212284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.213036 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:54.214613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:54.226161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1131/2296] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:04:54.230203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:54.240979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b27557-8cde-4554-9c2c-0d3681101f76 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:54.245656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bd869-c3cd-4e7c-ab01-6f241d2f5e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:54.246320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd869-c3cd-4e7c-ab01-6f241d2f5e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.246597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd869-c3cd-4e7c-ab01-6f241d2f5e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.246866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd869-c3cd-4e7c-ab01-6f241d2f5e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.247579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bd869-c3cd-4e7c-ab01-6f241d2f5e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:54.249249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bd869-c3cd-4e7c-ab01-6f241d2f5e30 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:54.259390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1166/2297] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:04:54.263597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:54.273650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f127-234c-4c66-8bbd-054a63271b56 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:54.278226 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:54.279131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.279412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.279688 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] PATH_INFO: `/v3/projects/433521f02a754082914aaf5fbe3d87cd/users/bc1a154fdd7a431c8978a170dc2d68c9/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.295268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=433521f02a754082914aaf5fbe3d87cd, user_id=bc1a154fdd7a431c8978a170dc2d68c9, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:54.298577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:54.307289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:54.313120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1132/2298] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:04:54 2022] PUT /identity/v3/projects/433521f02a754082914aaf5fbe3d87cd/users/bc1a154fdd7a431c8978a170dc2d68c9/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:04:54.323417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.323705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.323984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.355554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] MFA Rules not processed for user `bc1a154fdd7a431c8978a170dc2d68c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:04:54.356961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:54.423751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1167/2299] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:04:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:04:54.434788 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:54.436662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:54.515043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52d63-fcaa-43aa-b239-41bb0fff0ac8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:54.520956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:54.521599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:54.521880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:54.522150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:54.530289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:54.532404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:54.541384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1133/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:04:56.799641 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:04:56.801118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:56.854951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:04:56.860226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e36da-e00e-4a7d-ab81-1056b46d384a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:04:56.865179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:04:56.866064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:04:56.866422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:04:56.866692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:04:56.868482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:04:56.936172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:04:56.938307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:04:56.948596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1168/2301] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:00.099387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.100914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:00.119687 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730e99-933d-447f-808c-9176dabf9b58 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.124157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.125264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.125543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.125817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] PATH_INFO: `/v3/users/e9ca987fc23d4ff1a88dd4ed7b38980c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.126990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ca987fc23d4ff1a88dd4ed7b38980c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:00.128959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:00.176188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ca987fc23d4ff1a88dd4ed7b38980c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.176919 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ca987fc23d4ff1a88dd4ed7b38980c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.183662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ca987fc23d4ff1a88dd4ed7b38980c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.184236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ca987fc23d4ff1a88dd4ed7b38980c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.192065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-61f7db16-1d28-419e-94c8-b50c2c1d4018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ca987fc23d4ff1a88dd4ed7b38980c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.209933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1134/2302] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:00 2022] DELETE /identity/v3/users/e9ca987fc23d4ff1a88dd4ed7b38980c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:00.387624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.407037 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea1dc2-02be-4c52-b4a3-49b52fd67086 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.469918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.471105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.471530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.471817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] PATH_INFO: `/v3/projects/960d9a6b1f514344aa82bbada2b878ca` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.473040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=960d9a6b1f514344aa82bbada2b878ca)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:00.475000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:00.524670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960d9a6b1f514344aa82bbada2b878ca', 'request_id': 'req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.530122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960d9a6b1f514344aa82bbada2b878ca', 'request_id': 'req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.537137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960d9a6b1f514344aa82bbada2b878ca', 'request_id': 'req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.551560 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] The token cache is being invalidate because project 960d9a6b1f514344aa82bbada2b878ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:00.551853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.552518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-832c304e-9fbd-4ce1-8a7f-426c40a60fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:00.554079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1169/2303] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:00 2022] DELETE /identity/v3/projects/960d9a6b1f514344aa82bbada2b878ca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:00.559667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663f36-7994-4abd-a9c2-4df9fbe84cc8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.559957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663f36-7994-4abd-a9c2-4df9fbe84cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.561042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663f36-7994-4abd-a9c2-4df9fbe84cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.583835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-13663f36-7994-4abd-a9c2-4df9fbe84cc8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:00.585505 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-13663f36-7994-4abd-a9c2-4df9fbe84cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:00.656393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:00.668406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1135/2304] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:00.676990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e6556-3eb9-4bf3-a8c0-6ade31e158e8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.677606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e6556-3eb9-4bf3-a8c0-6ade31e158e8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.678096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e6556-3eb9-4bf3-a8c0-6ade31e158e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.700697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-ae1e6556-3eb9-4bf3-a8c0-6ade31e158e8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:00.702068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ae1e6556-3eb9-4bf3-a8c0-6ade31e158e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:00.710745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1170/2305] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:00.714908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663f36-7994-4abd-a9c2-4df9fbe84cc8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.726052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663f36-7994-4abd-a9c2-4df9fbe84cc8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.731692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.732437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.732724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.732997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.734299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:00.734836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:00.736920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:00.755019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1136/2306] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:00.756430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e6556-3eb9-4bf3-a8c0-6ade31e158e8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.766319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e6556-3eb9-4bf3-a8c0-6ade31e158e8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.771568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.772223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.772540 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.772819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.773522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:00.775497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:00.795445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:00.804391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1171/2307] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:00 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:00.807409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.818243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eef4f-164b-4b67-afd1-0f05b7ea2d67 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.873851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.875408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.875671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.875945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.876660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:00.878702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:00.898546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1137/2308] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:00 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:00.902741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.913942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3194-2c7b-413e-8d83-627cb265f06b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.918656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.919316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.919598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.919872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.920633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:00.922823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:00.935129 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1172/2309] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:00.940071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.950565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384b297-c081-412e-84ee-55a27f22b9ec admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.956732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74674ec3-5401-41e1-b6e2-570636487fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.957128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74674ec3-5401-41e1-b6e2-570636487fce admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.957415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74674ec3-5401-41e1-b6e2-570636487fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.957695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74674ec3-5401-41e1-b6e2-570636487fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:00.958717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74674ec3-5401-41e1-b6e2-570636487fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:00.960562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74674ec3-5401-41e1-b6e2-570636487fce admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:00.972612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1138/2310] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:00.976731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:00.987851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fc75c-a8df-4127-9335-235f68425651 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:00.992537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:00.993400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:00.993680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:00.993950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] PATH_INFO: `/v3/projects/6ec161693e7e4cb2987fe2ea34481197/users/31349552132d462a97d24b609218051f/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.007938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec161693e7e4cb2987fe2ea34481197, user_id=31349552132d462a97d24b609218051f, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.010592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.023138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1173/2311] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:00 2022] PUT /identity/v3/projects/6ec161693e7e4cb2987fe2ea34481197/users/31349552132d462a97d24b609218051f/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:01.033487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868a1a-aa36-4dc5-b644-a3cdbc5c3804 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.033775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868a1a-aa36-4dc5-b644-a3cdbc5c3804 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.034051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868a1a-aa36-4dc5-b644-a3cdbc5c3804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.061117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a3868a1a-aa36-4dc5-b644-a3cdbc5c3804 None None] MFA Rules not processed for user `31349552132d462a97d24b609218051f`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:01.062442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a3868a1a-aa36-4dc5-b644-a3cdbc5c3804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:01.124380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1139/2312] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:01.128843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.191244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0e3e3-e644-489a-8b7f-7fda39d83cbb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.195805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.196476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.196761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.197033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.197720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.199612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.224728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1174/2313] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:01 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:01.228684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3868a1a-aa36-4dc5-b644-a3cdbc5c3804 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.239586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3868a1a-aa36-4dc5-b644-a3cdbc5c3804 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.286902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:01.297646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.298311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.298595 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.298870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.299555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.301500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.321015 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1140/2314] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:01 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:01.324968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.335362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9be95f-c8b8-47f4-8c83-2a2fe9f36765 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.339983 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.340664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.341029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.341216 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.341936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.343631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.354436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1175/2315] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:01.359214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.370312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb09959-c278-48af-8587-fdc2ee740084 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.375565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39c230-5a95-428a-abae-636460a487bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.376284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39c230-5a95-428a-abae-636460a487bf admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.377327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39c230-5a95-428a-abae-636460a487bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.377616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39c230-5a95-428a-abae-636460a487bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.378347 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39c230-5a95-428a-abae-636460a487bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.379956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39c230-5a95-428a-abae-636460a487bf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.390255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1141/2316] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:01.394520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.404630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cda78-78b9-4d92-a6f5-26ebc65c1731 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.409270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.410143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.410424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.410697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] PATH_INFO: `/v3/projects/39f030b654764dc6b1ce60e6152a9f7d/users/ebe7a2b03fda405490b37b01b8a92844/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.425903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f030b654764dc6b1ce60e6152a9f7d, user_id=ebe7a2b03fda405490b37b01b8a92844, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.428579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.442653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1176/2317] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:01 2022] PUT /identity/v3/projects/39f030b654764dc6b1ce60e6152a9f7d/users/ebe7a2b03fda405490b37b01b8a92844/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:01.454641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be33618-5177-4548-b13a-16ffcccd79c5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.454930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be33618-5177-4548-b13a-16ffcccd79c5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.455264 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be33618-5177-4548-b13a-16ffcccd79c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.482012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4be33618-5177-4548-b13a-16ffcccd79c5 None None] MFA Rules not processed for user `ebe7a2b03fda405490b37b01b8a92844`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:01.483202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4be33618-5177-4548-b13a-16ffcccd79c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:01.545174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1142/2318] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:01.550236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.610570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b4415-4885-4023-889f-2b5d0c4cee0b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.615288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.615951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.616234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.616543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.617244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.619132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.640493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:01.648753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1177/2319] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:01 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:01.652689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be33618-5177-4548-b13a-16ffcccd79c5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.664327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be33618-5177-4548-b13a-16ffcccd79c5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.720673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.721459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.721750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.722029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.722729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.724683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.746145 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1143/2320] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:01 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:01.750649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.761095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7106c0-ae36-4be8-b75e-f5bc69f780ba admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.766556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.767346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.767624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.767895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.768659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.770248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.781229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1178/2321] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:01.785600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.808217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856bcd8-a012-4bf3-8e5a-c99b9e73f842 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.813566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.814463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.814748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.815023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] PATH_INFO: `/v3/projects/0d1583afcf2a42b78f7feaa4a8211714/users/5874e8f9bfd646e0a79214e48440f2a8/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.834817 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1583afcf2a42b78f7feaa4a8211714, user_id=5874e8f9bfd646e0a79214e48440f2a8, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.838019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.870578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1144/2322] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:01 2022] PUT /identity/v3/projects/0d1583afcf2a42b78f7feaa4a8211714/users/5874e8f9bfd646e0a79214e48440f2a8/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:01.871626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.883457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a71b8b-3168-4f40-8111-0cc056755225 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:01.954242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.954760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:01.967686 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:01.968793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:01.968951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:01.969231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:01.969966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:01.971554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:01.985914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1179/2323] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:01.989902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:01.994628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:02.000433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6d8a1-2419-4eb5-b9b2-12a67d72cf5c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:02.005665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11eb1e5-01ef-4ced-90e2-f8f425c1c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:02.006537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb1e5-01ef-4ced-90e2-f8f425c1c65c admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.006818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb1e5-01ef-4ced-90e2-f8f425c1c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.007091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb1e5-01ef-4ced-90e2-f8f425c1c65c admin admin] PATH_INFO: `/v3/projects/0d1583afcf2a42b78f7feaa4a8211714/users/5874e8f9bfd646e0a79214e48440f2a8/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.009085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11eb1e5-01ef-4ced-90e2-f8f425c1c65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1583afcf2a42b78f7feaa4a8211714, user_id=5874e8f9bfd646e0a79214e48440f2a8, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:02.011720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11eb1e5-01ef-4ced-90e2-f8f425c1c65c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:02.022799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1180/2324] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:01 2022] PUT /identity/v3/projects/0d1583afcf2a42b78f7feaa4a8211714/users/5874e8f9bfd646e0a79214e48440f2a8/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:02.031332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c2bbc-5eaf-42ba-b426-7841ec8f4e32 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:02.032627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.032916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.033196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.062414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] MFA Rules not processed for user `5874e8f9bfd646e0a79214e48440f2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:02.063697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.091184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:02.092068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.092383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.092652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] PATH_INFO: `/v3/users/bc1a154fdd7a431c8978a170dc2d68c9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.094046 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc1a154fdd7a431c8978a170dc2d68c9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:02.096059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:02.137151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1181/2325] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:02.150477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:02.151858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.159832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc1a154fdd7a431c8978a170dc2d68c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:02.160630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc1a154fdd7a431c8978a170dc2d68c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:02.167870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc1a154fdd7a431c8978a170dc2d68c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:02.168565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc1a154fdd7a431c8978a170dc2d68c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:02.176896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b1d654e0-68a3-4ae1-9e1f-4b93233cea01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc1a154fdd7a431c8978a170dc2d68c9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:02.195908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1145/2326] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:01 2022] DELETE /identity/v3/users/bc1a154fdd7a431c8978a170dc2d68c9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:02.201955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.202244 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.202559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.220875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:02.221093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:02.222887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.238102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b821e2-8327-434e-a128-9e8ae5025136 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:02.273160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:02.287287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1146/2327] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:02.289013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:02.289664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.289944 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.290214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.293159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.297613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:02.299215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.361246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:02.362704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:02.367365 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0003b-867e-4933-9fcf-9d3189f6fba2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:02.371437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:02.372225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.372537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.372811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.373817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1182/2328] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:02.383520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:02.384975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:02.396129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1147/2329] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:02.431103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:02.432707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.449838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f382f-bd59-4d0d-be16-cbaa54d4c6b1 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:02.455165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1588835-9646-4cf3-8660-349c188f36ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:02.456103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588835-9646-4cf3-8660-349c188f36ca service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.456499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588835-9646-4cf3-8660-349c188f36ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.456851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588835-9646-4cf3-8660-349c188f36ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.459457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b1588835-9646-4cf3-8660-349c188f36ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.540735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1588835-9646-4cf3-8660-349c188f36ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:02.542400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1588835-9646-4cf3-8660-349c188f36ca service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:02.552956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1183/2330] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:02.871249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:02.872757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:02.886193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f289f-1c4c-4916-b586-9dc1aa5e891f service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:02.892322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:02.893362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:02.893643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:02.893917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] PATH_INFO: `/v3/projects/433521f02a754082914aaf5fbe3d87cd` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:02.895227 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=433521f02a754082914aaf5fbe3d87cd)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:02.897531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:02.980302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '433521f02a754082914aaf5fbe3d87cd', 'request_id': 'req-843bafd3-d086-4070-b20b-38cec06c84c4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:02.989196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '433521f02a754082914aaf5fbe3d87cd', 'request_id': 'req-843bafd3-d086-4070-b20b-38cec06c84c4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:02.998501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '433521f02a754082914aaf5fbe3d87cd', 'request_id': 'req-843bafd3-d086-4070-b20b-38cec06c84c4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:03.012017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] The token cache is being invalidate because project 433521f02a754082914aaf5fbe3d87cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:03.012309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:03.013024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-843bafd3-d086-4070-b20b-38cec06c84c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:03.014571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1148/2331] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:02 2022] DELETE /identity/v3/projects/433521f02a754082914aaf5fbe3d87cd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:03.021546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d7272-bc47-424d-b772-443f80d8ebcb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.021546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d7272-bc47-424d-b772-443f80d8ebcb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.021546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d7272-bc47-424d-b772-443f80d8ebcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.042868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-437d7272-bc47-424d-b772-443f80d8ebcb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:03.044099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-437d7272-bc47-424d-b772-443f80d8ebcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:03.097863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:03.122225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1184/2332] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:03.130780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79b050-d35c-4af7-8435-9bb12bdf2e45 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.131076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79b050-d35c-4af7-8435-9bb12bdf2e45 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.131327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79b050-d35c-4af7-8435-9bb12bdf2e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.151601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8f79b050-d35c-4af7-8435-9bb12bdf2e45 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:03.152863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8f79b050-d35c-4af7-8435-9bb12bdf2e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:03.162084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1149/2333] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:03.166437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d7272-bc47-424d-b772-443f80d8ebcb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.177142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d7272-bc47-424d-b772-443f80d8ebcb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.181669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.182338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.182619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.182891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.183607 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:03.183906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.185538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.199809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1185/2334] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:03.205462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79b050-d35c-4af7-8435-9bb12bdf2e45 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.215369 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79b050-d35c-4af7-8435-9bb12bdf2e45 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.220462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.221211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.221570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.221928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.222696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.224741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.256551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1150/2335] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:03 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:03.257703 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.268945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce0fed-b79e-4f4a-b8c4-c38a9eee93ed admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.350268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.350966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.351257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.351981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.352198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.354149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.383038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1186/2336] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:03 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:03.387018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.405667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f215a4-6ab8-4613-a40e-ffe6707def91 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.413907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.414607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.424631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.425024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.425830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.427470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.450284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.451102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1151/2337] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:03.461775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27082f84-0443-42ea-bd2a-a3a5ae391948 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.467087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650887c-c962-4677-8d84-4e6f88fe9cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.467808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650887c-c962-4677-8d84-4e6f88fe9cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.468116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650887c-c962-4677-8d84-4e6f88fe9cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.468483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650887c-c962-4677-8d84-4e6f88fe9cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.469571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650887c-c962-4677-8d84-4e6f88fe9cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.471226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650887c-c962-4677-8d84-4e6f88fe9cdd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.482833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1187/2338] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:03.487059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.498790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bee972-c8e3-4fd6-8f60-ba29a246bbd9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.505233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.506386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.506767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.507113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] PATH_INFO: `/v3/projects/b8e3c8139bac4b77ad0096898c73a5b2/users/8d91feb9b5fe471495fdf4f248f2029a/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.529345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e3c8139bac4b77ad0096898c73a5b2, user_id=8d91feb9b5fe471495fdf4f248f2029a, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.533311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.552295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1152/2339] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:03 2022] PUT /identity/v3/projects/b8e3c8139bac4b77ad0096898c73a5b2/users/8d91feb9b5fe471495fdf4f248f2029a/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:03.564282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19de8f-6900-40ba-aa55-9a006961fcaf None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.564611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19de8f-6900-40ba-aa55-9a006961fcaf None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.564907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19de8f-6900-40ba-aa55-9a006961fcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.603407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7d19de8f-6900-40ba-aa55-9a006961fcaf None None] MFA Rules not processed for user `8d91feb9b5fe471495fdf4f248f2029a`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:03.604890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7d19de8f-6900-40ba-aa55-9a006961fcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:03.674036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1188/2340] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:03.678600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.698443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:03.743166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35b43e-3505-46c4-b177-3d406c053b8c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.747962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.748665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.748951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.749306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.750067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.751958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.796143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1153/2341] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:03 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:03.797893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19de8f-6900-40ba-aa55-9a006961fcaf None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.808856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19de8f-6900-40ba-aa55-9a006961fcaf None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.847263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:03.877039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.877786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.878051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.878931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.879891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.883572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.914995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1189/2342] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:03 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:03.917891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.931127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ca291-7692-44b7-b249-ed4a9b612f62 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.938059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.938755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.939037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.939311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:03.940400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:03.942269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:03.957866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1154/2343] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:03.962640 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:03.979469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeb65-ab51-4e74-ac40-22552cc45a77 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:03.985122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:03.986066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:03.986450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:03.986837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] PATH_INFO: `/v3/projects/78f627bedf2c4c9cba0cd8b3fed41fc8/users/ab749bc5071c40af82b091151953e61d/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.010521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f627bedf2c4c9cba0cd8b3fed41fc8, user_id=ab749bc5071c40af82b091151953e61d, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:04.014113 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:04.029921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1190/2344] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:03 2022] PUT /identity/v3/projects/78f627bedf2c4c9cba0cd8b3fed41fc8/users/ab749bc5071c40af82b091151953e61d/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:04.033994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:04.044823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af5e36-1858-4db7-9a13-ba59e959f81d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:04.119611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b9f91-5642-4663-8f53-45a083675252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:04.120720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b9f91-5642-4663-8f53-45a083675252 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:04.121456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b9f91-5642-4663-8f53-45a083675252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:04.122412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b9f91-5642-4663-8f53-45a083675252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.123590 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b9f91-5642-4663-8f53-45a083675252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:04.126120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b9f91-5642-4663-8f53-45a083675252 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:04.155604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1155/2345] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:04.157521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:04.174653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aefe17-59c6-4a3f-be5e-97f0f70d9bc2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:04.184331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:04.185376 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:04.185725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:04.186231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] PATH_INFO: `/v3/projects/78f627bedf2c4c9cba0cd8b3fed41fc8/users/ab749bc5071c40af82b091151953e61d/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.189765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f627bedf2c4c9cba0cd8b3fed41fc8, user_id=ab749bc5071c40af82b091151953e61d, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:04.192562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:04.206570 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1191/2346] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:04 2022] PUT /identity/v3/projects/78f627bedf2c4c9cba0cd8b3fed41fc8/users/ab749bc5071c40af82b091151953e61d/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:04.217352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1721e99-5641-49fb-aa51-411ef45ed656 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:04.217839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1721e99-5641-49fb-aa51-411ef45ed656 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:04.219432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1721e99-5641-49fb-aa51-411ef45ed656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.269411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d1721e99-5641-49fb-aa51-411ef45ed656 None None] MFA Rules not processed for user `ab749bc5071c40af82b091151953e61d`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:04.270694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d1721e99-5641-49fb-aa51-411ef45ed656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:04.370555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1156/2347] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:04.382692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:04.388062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:04.488289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f127159-ab8a-4486-a07f-f5610ec603f6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:04.494964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fef9e-273a-4ac4-a0ba-2ce2c1a96b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:04.495952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fef9e-273a-4ac4-a0ba-2ce2c1a96b82 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:04.496229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fef9e-273a-4ac4-a0ba-2ce2c1a96b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:04.496539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fef9e-273a-4ac4-a0ba-2ce2c1a96b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.510954 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fef9e-273a-4ac4-a0ba-2ce2c1a96b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:04.513506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fef9e-273a-4ac4-a0ba-2ce2c1a96b82 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:04.527434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1192/2348] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:04.600153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1721e99-5641-49fb-aa51-411ef45ed656 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:04.604441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:04.695671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1721e99-5641-49fb-aa51-411ef45ed656 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:04.700989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:04.701982 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:04.702975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:04.703216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] PATH_INFO: `/v3/users/8d91feb9b5fe471495fdf4f248f2029a` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.704558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d91feb9b5fe471495fdf4f248f2029a)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:04.707346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:04.759809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d91feb9b5fe471495fdf4f248f2029a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:04.760490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d91feb9b5fe471495fdf4f248f2029a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:04.768087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d91feb9b5fe471495fdf4f248f2029a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:04.768882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d91feb9b5fe471495fdf4f248f2029a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:04.775964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d91feb9b5fe471495fdf4f248f2029a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:04.800940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1157/2349] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:04 2022] DELETE /identity/v3/users/8d91feb9b5fe471495fdf4f248f2029a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:04.805619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:04.805943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:04.806225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.825943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:04.827144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:04.889524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1193/2350] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:04.899836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:04.901262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:04.966634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620bfaf-4cb2-40c7-8161-016ed551ecd8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:04.970548 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:04.971191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:04.971474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:04.971749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:04.981826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:04.983248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:04.993292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1158/2351] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:05.262604 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:05.264180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:05.277410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083ded0-7f0b-4439-adf6-8369d611a10b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:05.282778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:05.283720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:05.284070 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:05.284449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] PATH_INFO: `/v3/projects/b8e3c8139bac4b77ad0096898c73a5b2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:05.285709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e3c8139bac4b77ad0096898c73a5b2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:05.287673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:05.320663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:05.336967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e3c8139bac4b77ad0096898c73a5b2', 'request_id': 'req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.342674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e3c8139bac4b77ad0096898c73a5b2', 'request_id': 'req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.349186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e3c8139bac4b77ad0096898c73a5b2', 'request_id': 'req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.362537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] The token cache is being invalidate because project b8e3c8139bac4b77ad0096898c73a5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:05.362819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.363424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.364965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1194/2352] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:05 2022] DELETE /identity/v3/projects/b8e3c8139bac4b77ad0096898c73a5b2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:05.369224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:05.380578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60102a01-cd91-4e27-9e99-2369c4e2d0af service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:05.403565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:05.447719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:05.448647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:05.448931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:05.449205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] PATH_INFO: `/v3/users/ab749bc5071c40af82b091151953e61d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:05.450422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab749bc5071c40af82b091151953e61d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:05.452495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:05.506479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab749bc5071c40af82b091151953e61d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.507389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab749bc5071c40af82b091151953e61d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.517658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab749bc5071c40af82b091151953e61d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.521445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab749bc5071c40af82b091151953e61d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.528149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8cb49732-63bf-4384-ae08-ab9c02c0f0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab749bc5071c40af82b091151953e61d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.548728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1159/2353] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:05 2022] DELETE /identity/v3/users/ab749bc5071c40af82b091151953e61d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:05.596090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:05.597838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:05.612302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeaa1dc-4ab0-4740-9d8d-878b7cf07bd4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:05.673666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:05.674548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:05.674830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:05.675106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] PATH_INFO: `/v3/users/31349552132d462a97d24b609218051f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:05.676290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31349552132d462a97d24b609218051f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:05.678324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:05.733091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31349552132d462a97d24b609218051f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.733973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31349552132d462a97d24b609218051f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.734588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31349552132d462a97d24b609218051f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.742127 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31349552132d462a97d24b609218051f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.756867 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31349552132d462a97d24b609218051f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:05.773204 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1195/2354] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:05 2022] DELETE /identity/v3/users/31349552132d462a97d24b609218051f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:05.777588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:05.777876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:05.778155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:05.795380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:05.797663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:05.849358 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:05.851616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:05.869379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1160/2355] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:05.879601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:05.881232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:05.907591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:05.930553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:05.938651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4212af-944a-4ce0-82fc-00d8a3e6c45f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:05.944655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:05.945266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e1e9-9534-438a-84c7-97ef3e014514 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:05.945529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:05.945807 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:05.946086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] PATH_INFO: `/v3/projects/78f627bedf2c4c9cba0cd8b3fed41fc8` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:05.947315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f627bedf2c4c9cba0cd8b3fed41fc8)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:05.949259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:05.949647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a429a-239b-4b65-9821-3eee16f2d1d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:05.950363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a429a-239b-4b65-9821-3eee16f2d1d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:05.950703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a429a-239b-4b65-9821-3eee16f2d1d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:05.951043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a429a-239b-4b65-9821-3eee16f2d1d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:05.961609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a429a-239b-4b65-9821-3eee16f2d1d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:05.963181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a429a-239b-4b65-9821-3eee16f2d1d6 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:05.974979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1161/2356] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:06.000647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f627bedf2c4c9cba0cd8b3fed41fc8', 'request_id': 'req-7b73321c-094b-48dc-864e-934987c249b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.007092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f627bedf2c4c9cba0cd8b3fed41fc8', 'request_id': 'req-7b73321c-094b-48dc-864e-934987c249b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.013602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f627bedf2c4c9cba0cd8b3fed41fc8', 'request_id': 'req-7b73321c-094b-48dc-864e-934987c249b2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.027691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] The token cache is being invalidate because project 78f627bedf2c4c9cba0cd8b3fed41fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:06.027946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.028900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-7b73321c-094b-48dc-864e-934987c249b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.031378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1196/2357] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:05 2022] DELETE /identity/v3/projects/78f627bedf2c4c9cba0cd8b3fed41fc8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:06.037060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c05f53-10a1-401f-a92f-7ecc4eb3ed90 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.037408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c05f53-10a1-401f-a92f-7ecc4eb3ed90 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.037799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c05f53-10a1-401f-a92f-7ecc4eb3ed90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.057741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-86c05f53-10a1-401f-a92f-7ecc4eb3ed90 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:06.058929 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-86c05f53-10a1-401f-a92f-7ecc4eb3ed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.156511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1162/2358] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:06.160078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc8b53-8c95-47bd-9ef5-7c64d021b9a1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.161295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc8b53-8c95-47bd-9ef5-7c64d021b9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.161603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc8b53-8c95-47bd-9ef5-7c64d021b9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.181220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-72cc8b53-8c95-47bd-9ef5-7c64d021b9a1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:06.182459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-72cc8b53-8c95-47bd-9ef5-7c64d021b9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.198993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1197/2359] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:06.203578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c05f53-10a1-401f-a92f-7ecc4eb3ed90 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.213737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c05f53-10a1-401f-a92f-7ecc4eb3ed90 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.219374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.220108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.220562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.220907 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.221706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:06.222082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.223755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.237279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1163/2360] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:06.241629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc8b53-8c95-47bd-9ef5-7c64d021b9a1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.252172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc8b53-8c95-47bd-9ef5-7c64d021b9a1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.257432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.258097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.258382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.258675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.259602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.261704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.283414 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.285838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.302063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1198/2361] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:06 2022] POST /identity/v3/projects => generated 393 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:06.305341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3ae12-81d0-49b5-ad92-eeaf8390cfb9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.306035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.379833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.381321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.381605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.381879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] PATH_INFO: `/v3/projects/6ec161693e7e4cb2987fe2ea34481197` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.383016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6ab5-c9f1-4c6e-a04b-0863671dc694 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.383130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec161693e7e4cb2987fe2ea34481197)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.385113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.388630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.389297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.389577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.390588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.391367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.393297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.415233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1199/2362] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:06 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:06.420628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.430976 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b63c66-3fd5-4d33-bdbf-a0947ae8cd6c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.438253 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec161693e7e4cb2987fe2ea34481197', 'request_id': 'req-449b68a6-efff-4295-80ab-6624917e9484'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.446107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec161693e7e4cb2987fe2ea34481197', 'request_id': 'req-449b68a6-efff-4295-80ab-6624917e9484'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.455469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec161693e7e4cb2987fe2ea34481197', 'request_id': 'req-449b68a6-efff-4295-80ab-6624917e9484'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.468876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:06.473332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] The token cache is being invalidate because project 6ec161693e7e4cb2987fe2ea34481197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:06.473643 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.474441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.478112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1164/2363] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:06 2022] DELETE /identity/v3/projects/6ec161693e7e4cb2987fe2ea34481197 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:06.481150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.483993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.492992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:06.511254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.512343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.512652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.512999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.514549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.516863 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.527476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1200/2364] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:06.532546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.534228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.548025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163ebc2-b80f-47a2-86aa-ed6c39f29d77 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.554286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.555242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.555525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.555797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.556548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.558108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.570269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1201/2365] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:06.571273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b68a6-efff-4295-80ab-6624917e9484 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.574243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.576130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.577058 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.577384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.577675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] PATH_INFO: `/v3/users/ebe7a2b03fda405490b37b01b8a92844` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.578886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe7a2b03fda405490b37b01b8a92844)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.580846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.585287 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b52ea7-6894-433f-9a2d-235d90139e9a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.590874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2276b7f-5984-4464-9306-a762a4cd813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.591763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276b7f-5984-4464-9306-a762a4cd813c admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.592044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276b7f-5984-4464-9306-a762a4cd813c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.592311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276b7f-5984-4464-9306-a762a4cd813c admin admin] PATH_INFO: `/v3/projects/440931d1288d4a548a0b6c2de99fc029/users/668d2d32ec0d4092af48f856d62739a2/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.607803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2276b7f-5984-4464-9306-a762a4cd813c admin admin] RBAC: Authorizing `identity:create_grant(project_id=440931d1288d4a548a0b6c2de99fc029, user_id=668d2d32ec0d4092af48f856d62739a2, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.611224 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2276b7f-5984-4464-9306-a762a4cd813c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.623023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1202/2366] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:06 2022] PUT /identity/v3/projects/440931d1288d4a548a0b6c2de99fc029/users/668d2d32ec0d4092af48f856d62739a2/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:06.630418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe7a2b03fda405490b37b01b8a92844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.631103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe7a2b03fda405490b37b01b8a92844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.635527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c0794-893e-4ed3-968a-79931f593b16 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.635887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c0794-893e-4ed3-968a-79931f593b16 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.636240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c0794-893e-4ed3-968a-79931f593b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.638991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe7a2b03fda405490b37b01b8a92844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.639628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe7a2b03fda405490b37b01b8a92844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.648620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe7a2b03fda405490b37b01b8a92844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:06.662160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-023c0794-893e-4ed3-968a-79931f593b16 None None] MFA Rules not processed for user `668d2d32ec0d4092af48f856d62739a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:06.663362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-023c0794-893e-4ed3-968a-79931f593b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.665883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1165/2367] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:06 2022] DELETE /identity/v3/users/ebe7a2b03fda405490b37b01b8a92844 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:06.730947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1203/2368] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:06.736792 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.738213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.817701 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89030f4d-76db-4e21-b242-2bf7e015e8ee admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.822940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.823614 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.823900 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.824182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.824999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.826990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.857345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1166/2369] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:06 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:06.859079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c0794-893e-4ed3-968a-79931f593b16 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.870338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c0794-893e-4ed3-968a-79931f593b16 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.928746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.929491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.929835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.930111 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.930797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.932900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.960177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.964017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1204/2370] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:06 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:06.964170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:06.965386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:06.976460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b1c36-64e8-40ae-8d84-2e85d338ddbd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.977161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88915bb-4db7-4137-9171-a5db3267b2b8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:06.981848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.982158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.982272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:06.982428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.982705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.983030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:06.983315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:06.983502 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.984200 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] PATH_INFO: `/v3/projects/39f030b654764dc6b1ce60e6152a9f7d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:06.985582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f030b654764dc6b1ce60e6152a9f7d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:06.985901 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.987489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:06.987664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:06.990642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:07.002198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1205/2371] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:07.002931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.013784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8422b75-74ab-48ff-8996-10c3d9bbc753 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.019748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.020726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.021118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.021563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] PATH_INFO: `/v3/projects/94256ecd09ec49498dc6b9f6c0b21c46/users/ab0b190d616e42a0a658e5c859f74521/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.037995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94256ecd09ec49498dc6b9f6c0b21c46, user_id=ab0b190d616e42a0a658e5c859f74521, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.040773 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.042313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f030b654764dc6b1ce60e6152a9f7d', 'request_id': 'req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.051125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f030b654764dc6b1ce60e6152a9f7d', 'request_id': 'req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.056090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1206/2372] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:07 2022] PUT /identity/v3/projects/94256ecd09ec49498dc6b9f6c0b21c46/users/ab0b190d616e42a0a658e5c859f74521/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:07.059897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.061446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f030b654764dc6b1ce60e6152a9f7d', 'request_id': 'req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.073071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] The token cache is being invalidate because project 39f030b654764dc6b1ce60e6152a9f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:07.073317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.074006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.075579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1167/2373] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:06 2022] DELETE /identity/v3/projects/39f030b654764dc6b1ce60e6152a9f7d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:07.079474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.080924 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.124167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216883-b9b4-4a93-8475-0ecc486ade14 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.129829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.130498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.130779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.131055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.131782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.133489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.144387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1207/2374] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:07.149292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.151116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.151988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2f7fc-12b2-45be-9e8a-bf7ed77e78e0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.157191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.158065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.158343 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.158616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] PATH_INFO: `/v3/users/5874e8f9bfd646e0a79214e48440f2a8` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.159818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5874e8f9bfd646e0a79214e48440f2a8)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.161751 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.164281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c7a3-4f0a-4df9-82cb-e17fe1f7e4f0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.169046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35862346-784e-4e49-9672-cd1a7a35fc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.169957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35862346-784e-4e49-9672-cd1a7a35fc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.170275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35862346-784e-4e49-9672-cd1a7a35fc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.170552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35862346-784e-4e49-9672-cd1a7a35fc84 admin admin] PATH_INFO: `/v3/projects/94256ecd09ec49498dc6b9f6c0b21c46/users/ab0b190d616e42a0a658e5c859f74521/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.173913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35862346-784e-4e49-9672-cd1a7a35fc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94256ecd09ec49498dc6b9f6c0b21c46, user_id=ab0b190d616e42a0a658e5c859f74521, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.176612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35862346-784e-4e49-9672-cd1a7a35fc84 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.187698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1208/2375] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:07 2022] PUT /identity/v3/projects/94256ecd09ec49498dc6b9f6c0b21c46/users/ab0b190d616e42a0a658e5c859f74521/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:07.198744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff817bd-e5d4-4b59-9b2d-d57dbbfdf43f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.199265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff817bd-e5d4-4b59-9b2d-d57dbbfdf43f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.199408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff817bd-e5d4-4b59-9b2d-d57dbbfdf43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.208293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5874e8f9bfd646e0a79214e48440f2a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.209116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5874e8f9bfd646e0a79214e48440f2a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.216153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5874e8f9bfd646e0a79214e48440f2a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.216834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5874e8f9bfd646e0a79214e48440f2a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.225346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5874e8f9bfd646e0a79214e48440f2a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.227832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2ff817bd-e5d4-4b59-9b2d-d57dbbfdf43f None None] MFA Rules not processed for user `ab0b190d616e42a0a658e5c859f74521`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:07.229098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2ff817bd-e5d4-4b59-9b2d-d57dbbfdf43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.243593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1168/2376] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:07 2022] DELETE /identity/v3/users/5874e8f9bfd646e0a79214e48440f2a8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:07.302671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1209/2377] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:07.314344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.316128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.330928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:07.385238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877097e-dc62-43cc-806b-6dfa53f30866 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.390869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.391529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.391874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.392093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.488573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.490340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.503065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1169/2378] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:07.554747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff817bd-e5d4-4b59-9b2d-d57dbbfdf43f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.581144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.582855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.620465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:07.638040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff817bd-e5d4-4b59-9b2d-d57dbbfdf43f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.641974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834934-062b-4d00-a676-eb4fa54fcad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.642838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834934-062b-4d00-a676-eb4fa54fcad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.643425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834934-062b-4d00-a676-eb4fa54fcad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.643714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834934-062b-4d00-a676-eb4fa54fcad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.645536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cd834934-062b-4d00-a676-eb4fa54fcad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.672936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3549f-95aa-4755-b18e-6d42228b42cc service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.677709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.678598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.678878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.679175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] PATH_INFO: `/v3/projects/0d1583afcf2a42b78f7feaa4a8211714` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.680416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1583afcf2a42b78f7feaa4a8211714)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.682381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.706911 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834934-062b-4d00-a676-eb4fa54fcad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.708409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834934-062b-4d00-a676-eb4fa54fcad6 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.718907 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1210/2379] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:07.734435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1583afcf2a42b78f7feaa4a8211714', 'request_id': 'req-df12bb72-0f7c-4166-808f-8a066459916a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.740941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1583afcf2a42b78f7feaa4a8211714', 'request_id': 'req-df12bb72-0f7c-4166-808f-8a066459916a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.748765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1583afcf2a42b78f7feaa4a8211714', 'request_id': 'req-df12bb72-0f7c-4166-808f-8a066459916a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.751029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:07.763373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] The token cache is being invalidate because project 0d1583afcf2a42b78f7feaa4a8211714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:07.763663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.764336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-df12bb72-0f7c-4166-808f-8a066459916a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:07.766619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1170/2380] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:07 2022] DELETE /identity/v3/projects/0d1583afcf2a42b78f7feaa4a8211714 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:07.772743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11401-7da7-4fa4-ae13-b44a695dfe1a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.773136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11401-7da7-4fa4-ae13-b44a695dfe1a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.773481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11401-7da7-4fa4-ae13-b44a695dfe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.792543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9dc11401-7da7-4fa4-ae13-b44a695dfe1a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:07.793741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9dc11401-7da7-4fa4-ae13-b44a695dfe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.861869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1211/2381] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:07.870654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242b775-e7ff-4863-b314-46d581f3299c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.871009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242b775-e7ff-4863-b314-46d581f3299c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.871288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242b775-e7ff-4863-b314-46d581f3299c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.890734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c242b775-e7ff-4863-b314-46d581f3299c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:07.891949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c242b775-e7ff-4863-b314-46d581f3299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:07.900495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1171/2382] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:07.904701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11401-7da7-4fa4-ae13-b44a695dfe1a None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.915458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11401-7da7-4fa4-ae13-b44a695dfe1a None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.920263 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.921517 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.921923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.922080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.922859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:07.923178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.924802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.936660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1212/2383] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:07.941135 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242b775-e7ff-4863-b314-46d581f3299c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:07.950457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242b775-e7ff-4863-b314-46d581f3299c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:07.956813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:07.957768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:07.958086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:07.958381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:07.959221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:07.961706 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:07.994040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1172/2384] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:07 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:07.998361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:08.008946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d4892-0e19-482a-bfa6-709a61716381 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:08.071761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:08.072605 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:08.072891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:08.073166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:08.073899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:08.075845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:08.096707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1213/2385] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:07 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:08.100776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:08.111418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154a2ea-7460-4ce6-bc72-b6c090fa4e6f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:08.115838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:08.116526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:08.116805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:08.117079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:08.117820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:08.119396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:08.129798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1173/2386] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:08.133791 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:08.144681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4f0fd-3d5c-419c-9118-87de23380254 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:08.149400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1caf6f-c6c8-4077-a2a1-fd51de263e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:08.150068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1caf6f-c6c8-4077-a2a1-fd51de263e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:08.150349 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1caf6f-c6c8-4077-a2a1-fd51de263e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:08.150621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1caf6f-c6c8-4077-a2a1-fd51de263e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:08.151709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1caf6f-c6c8-4077-a2a1-fd51de263e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:08.152955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1caf6f-c6c8-4077-a2a1-fd51de263e2c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:08.158950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:08.165561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1214/2387] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:08.169636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:08.179772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57e20-d3d9-403d-a79a-c106d7006fb0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:08.184286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:08.185184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:08.185462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:08.185731 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] PATH_INFO: `/v3/projects/29fcd5de091e4a82ac71d8085be7144e/users/9bc58b7d56b340119a39ea7d86ee30ff/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:08.199710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fcd5de091e4a82ac71d8085be7144e, user_id=9bc58b7d56b340119a39ea7d86ee30ff, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:08.203000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:08.215259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1174/2388] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:08 2022] PUT /identity/v3/projects/29fcd5de091e4a82ac71d8085be7144e/users/9bc58b7d56b340119a39ea7d86ee30ff/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:08.227708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:08.227991 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:08.228264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:08.254623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] MFA Rules not processed for user `9bc58b7d56b340119a39ea7d86ee30ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:08.255799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:08.320375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1215/2389] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:08.332193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:08.333940 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:08.413344 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9efe7-ba93-45fd-bf40-aaf878ae2461 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:08.416993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0206e09e4f426d9f4b54543d89c039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:08.417629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:08.417909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:08.418183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:08.426445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:08.427869 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:08.436455 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1175/2390] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:08.760233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:08.761687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:08.838492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11234b-bf80-4360-b58a-0192728bb8c6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:08.842172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e10525-5d26-461e-8b13-4494a7a463a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:08.843479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10525-5d26-461e-8b13-4494a7a463a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:08.843870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10525-5d26-461e-8b13-4494a7a463a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:08.844153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10525-5d26-461e-8b13-4494a7a463a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:08.845979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e4e10525-5d26-461e-8b13-4494a7a463a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:08.914368 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e10525-5d26-461e-8b13-4494a7a463a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:08.916506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e10525-5d26-461e-8b13-4494a7a463a6 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:08.927133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1216/2391] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:11.121392 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:11.123172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:11.144124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa8b29-dd74-403c-8a87-1bd803a6e9c5 service glance] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:11.149376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:11.152205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:11.152604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:11.152955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] PATH_INFO: `/v3/users/668d2d32ec0d4092af48f856d62739a2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:11.154259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=668d2d32ec0d4092af48f856d62739a2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:11.159356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:11.217386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668d2d32ec0d4092af48f856d62739a2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:11.218543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668d2d32ec0d4092af48f856d62739a2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:11.233939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668d2d32ec0d4092af48f856d62739a2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:11.234702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668d2d32ec0d4092af48f856d62739a2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:11.248374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668d2d32ec0d4092af48f856d62739a2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:11.252156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:11.279229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1176/2392] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:11 2022] DELETE /identity/v3/users/668d2d32ec0d4092af48f856d62739a2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:11.283924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:11.284468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:11.284552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:11.302980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:11.304191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:11.344614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:11.372471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1217/2393] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:11.383316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:11.384750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:11.456609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e0f1-9144-4fd4-8ed3-3d5743292f8c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:11.460473 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:11.461130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:11.461407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:11.461676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:11.472553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:11.473976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:11.485213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1177/2394] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:11.928932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:11.930769 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:11.972604 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4256f22-e42e-4b64-89f8-00a4e53ef757 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:11.978195 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:11.979196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:11.979542 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:11.979969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] PATH_INFO: `/v3/projects/440931d1288d4a548a0b6c2de99fc029` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:11.981259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=440931d1288d4a548a0b6c2de99fc029)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:11.983662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:12.044480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440931d1288d4a548a0b6c2de99fc029', 'request_id': 'req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.055898 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440931d1288d4a548a0b6c2de99fc029', 'request_id': 'req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.063498 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440931d1288d4a548a0b6c2de99fc029', 'request_id': 'req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.093896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] The token cache is being invalidate because project 440931d1288d4a548a0b6c2de99fc029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:12.093896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.094705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.099509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1218/2395] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:11 2022] DELETE /identity/v3/projects/440931d1288d4a548a0b6c2de99fc029 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:12.101082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:12.122715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0782d9f-ee38-4c3c-a6db-573814f07555 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:12.215838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:12.216891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:12.217238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:12.217622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] PATH_INFO: `/v3/users/ab0b190d616e42a0a658e5c859f74521` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:12.219221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0b190d616e42a0a658e5c859f74521)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:12.221499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:12.304027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0b190d616e42a0a658e5c859f74521'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.305183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0b190d616e42a0a658e5c859f74521'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.318586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0b190d616e42a0a658e5c859f74521'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.319335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0b190d616e42a0a658e5c859f74521'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.339551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0b190d616e42a0a658e5c859f74521'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.379261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1178/2396] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:12 2022] DELETE /identity/v3/users/ab0b190d616e42a0a658e5c859f74521 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:12.709675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:12.711350 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:12.725845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4e8ee-ccd9-4dba-9cab-8f0dedceaa7e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:12.806666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:12.807561 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:12.807838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:12.808161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] PATH_INFO: `/v3/users/f47acc995d80476cbd4153e9121e97eb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:12.809410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47acc995d80476cbd4153e9121e97eb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:12.812077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:12.847202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:12.848984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:12.865467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:12.869401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4cc5-91cb-4435-a7df-a14987351e59 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:12.878613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47acc995d80476cbd4153e9121e97eb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.879490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:12.879655 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47acc995d80476cbd4153e9121e97eb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.880508 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47acc995d80476cbd4153e9121e97eb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.882089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:12.887246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:12.887246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] PATH_INFO: `/v3/projects/94256ecd09ec49498dc6b9f6c0b21c46` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:12.887884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94256ecd09ec49498dc6b9f6c0b21c46)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:12.890579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:12.895916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47acc995d80476cbd4153e9121e97eb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.904217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:12.910222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f21b97fa-abea-403b-b7f8-bc2c09fcd943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47acc995d80476cbd4153e9121e97eb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.935881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1219/2397] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:12 2022] DELETE /identity/v3/users/f47acc995d80476cbd4153e9121e97eb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:12.963397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94256ecd09ec49498dc6b9f6c0b21c46', 'request_id': 'req-6091c283-6518-42b7-a0fc-4511c5792ce1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.970978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94256ecd09ec49498dc6b9f6c0b21c46', 'request_id': 'req-6091c283-6518-42b7-a0fc-4511c5792ce1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.982655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94256ecd09ec49498dc6b9f6c0b21c46', 'request_id': 'req-6091c283-6518-42b7-a0fc-4511c5792ce1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.994550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] The token cache is being invalidate because project 94256ecd09ec49498dc6b9f6c0b21c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:12.994916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.995549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6091c283-6518-42b7-a0fc-4511c5792ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:12.997127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1179/2398] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:12 2022] DELETE /identity/v3/projects/94256ecd09ec49498dc6b9f6c0b21c46 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:13.002017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.002343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.002654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.021279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:13.022541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.089228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1220/2399] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:13.100413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045a307-02a9-4c40-bbc2-b8fc7b05ed71 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.100413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045a307-02a9-4c40-bbc2-b8fc7b05ed71 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.100413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045a307-02a9-4c40-bbc2-b8fc7b05ed71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.121138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6045a307-02a9-4c40-bbc2-b8fc7b05ed71 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:13.122667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6045a307-02a9-4c40-bbc2-b8fc7b05ed71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.137694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1180/2400] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:13.139268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.141167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.141704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045a307-02a9-4c40-bbc2-b8fc7b05ed71 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.152066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045a307-02a9-4c40-bbc2-b8fc7b05ed71 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.154334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321284-c177-431c-bd6c-12fb78af689d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.159068 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.159853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.160000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.160210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.160531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.161293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.161293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.161497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:13.161612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.163172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.164629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] PATH_INFO: `/v3/projects/e601e41f827a4c6f8c743a174ac463fa` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.166322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e601e41f827a4c6f8c743a174ac463fa)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.168419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.177850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1181/2401] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 16:05:13.182232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.193717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df2422-82eb-487a-a2cc-178c54fadb95 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.198580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.199647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.199991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.200493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.201282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.203748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.222162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e601e41f827a4c6f8c743a174ac463fa', 'request_id': 'req-322a0f10-505f-477e-95e7-489f295cbb44'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:13.228603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e601e41f827a4c6f8c743a174ac463fa', 'request_id': 'req-322a0f10-505f-477e-95e7-489f295cbb44'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:13.232761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1182/2402] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:13.237729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.239191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e601e41f827a4c6f8c743a174ac463fa', 'request_id': 'req-322a0f10-505f-477e-95e7-489f295cbb44'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:13.253307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] The token cache is being invalidate because project e601e41f827a4c6f8c743a174ac463fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:13.253599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:13.254273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-322a0f10-505f-477e-95e7-489f295cbb44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:13.256136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1221/2403] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:13 2022] DELETE /identity/v3/projects/e601e41f827a4c6f8c743a174ac463fa => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:13.260795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc35694-a9bc-4d9d-8b09-cf79f2bb63f1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.261082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc35694-a9bc-4d9d-8b09-cf79f2bb63f1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.261638 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc35694-a9bc-4d9d-8b09-cf79f2bb63f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.281833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4dc35694-a9bc-4d9d-8b09-cf79f2bb63f1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:13.283381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4dc35694-a9bc-4d9d-8b09-cf79f2bb63f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.305086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13e9b6-f665-4bbf-a608-a66a1035e6c5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.309724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.310385 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.310733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.311081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.311795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.313744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.337779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1183/2404] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:13.338966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.342559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.354038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1222/2405] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:13.362180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde72ddb-95a6-44e2-afb2-8118a7d08fc8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.362438 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a16540-e0b8-4a8f-9470-4f157e1489d0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.362549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde72ddb-95a6-44e2-afb2-8118a7d08fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.362741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde72ddb-95a6-44e2-afb2-8118a7d08fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.367244 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.367913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.368198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.368647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.369382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.375054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.380219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:13.385471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-bde72ddb-95a6-44e2-afb2-8118a7d08fc8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:13.387868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1184/2406] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:13.388004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-bde72ddb-95a6-44e2-afb2-8118a7d08fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.393256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.401675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1223/2407] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:13.407916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde72ddb-95a6-44e2-afb2-8118a7d08fc8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.410037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c91549-8d50-4e2d-86aa-44fd9583f843 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.416206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.416914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.417194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.417467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.418189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.419778 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.420406 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde72ddb-95a6-44e2-afb2-8118a7d08fc8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.426728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.427408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.427704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.427979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.429057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:13.429384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.430993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.434681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1185/2408] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:13.435488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.446587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224f711-26de-4001-bca9-3e17f6f8bb47 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.449047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1224/2409] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:13.449917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.451306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6f0ae-b837-4db5-a41b-e943a942b901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.453694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f0ae-b837-4db5-a41b-e943a942b901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.453978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f0ae-b837-4db5-a41b-e943a942b901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.454254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f0ae-b837-4db5-a41b-e943a942b901 admin admin] PATH_INFO: `/v3/projects/d879eaf8a03f45c2b1d8b5f925f80901/users/36bbee8799084903bf70ccb387a0244d/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.463210 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07912a95-9e25-4dae-ba5c-5294c45ac507 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.467098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:13.471644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.472828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.473091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.473272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6f0ae-b837-4db5-a41b-e943a942b901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d879eaf8a03f45c2b1d8b5f925f80901, user_id=36bbee8799084903bf70ccb387a0244d, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.473379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.473958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.475839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.476624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6f0ae-b837-4db5-a41b-e943a942b901 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.495237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1186/2410] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:13 2022] PUT /identity/v3/projects/d879eaf8a03f45c2b1d8b5f925f80901/users/36bbee8799084903bf70ccb387a0244d/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:13.504245 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972bbf2-f068-4b4f-bf40-86cdcb4f0e23 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.504573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972bbf2-f068-4b4f-bf40-86cdcb4f0e23 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.504723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1225/2411] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:13.504857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972bbf2-f068-4b4f-bf40-86cdcb4f0e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.508898 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.537043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4972bbf2-f068-4b4f-bf40-86cdcb4f0e23 None None] MFA Rules not processed for user `36bbee8799084903bf70ccb387a0244d`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:13.538256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4972bbf2-f068-4b4f-bf40-86cdcb4f0e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.572753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bebc04-3297-4318-af13-00a36bbddfb1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.578705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.579392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.579677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.579953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.580688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.582711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.611402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1226/2412] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/users => generated 279 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:13.612569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.613482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1187/2413] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:13.617777 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972bbf2-f068-4b4f-bf40-86cdcb4f0e23 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.626367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ddf66-e071-4be0-b179-19e3cc9ce98d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.628935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972bbf2-f068-4b4f-bf40-86cdcb4f0e23 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.631247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.631940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.632195 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.632514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.633243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.633601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.634251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.634556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.634830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.634929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.635564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.637489 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.646396 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1227/2414] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:13.650251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.661900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48910e-32ba-4f2c-9d9a-8229482d0f0b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.669219 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1188/2415] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:13.669386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.669386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.669386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.669386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.669745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.671009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.671665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.685379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1228/2416] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:13.690289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.700654 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f820e2-347c-4e5f-aa4f-74cc9651c592 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.742872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa5b9a-8f23-4a38-aa2a-7036ca6c840a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.747562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.748232 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.748556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.750064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.750064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.752070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.763507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da123649-1b12-407a-8a8f-3fcb51dafe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.764247 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da123649-1b12-407a-8a8f-3fcb51dafe79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.765248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da123649-1b12-407a-8a8f-3fcb51dafe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.765538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da123649-1b12-407a-8a8f-3fcb51dafe79 admin admin] PATH_INFO: `/v3/projects/27bd9552e272496d926c30ed3d5e591b/users/3cb2514bd57747a59a19ac2e2d392670/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.777368 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1189/2417] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:13.780969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.787242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da123649-1b12-407a-8a8f-3fcb51dafe79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bd9552e272496d926c30ed3d5e591b, user_id=3cb2514bd57747a59a19ac2e2d392670, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.787242 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da123649-1b12-407a-8a8f-3fcb51dafe79 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.792288 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480acee-b47b-4ed1-9493-52104bd3dcd8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.798781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.799451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.799737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.800017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.800790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.801199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1229/2418] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:13 2022] PUT /identity/v3/projects/27bd9552e272496d926c30ed3d5e591b/users/3cb2514bd57747a59a19ac2e2d392670/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:13.805570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.813105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3b824-e15e-48ad-a26c-9fa49fcbcc97 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.813395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3b824-e15e-48ad-a26c-9fa49fcbcc97 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.813676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3b824-e15e-48ad-a26c-9fa49fcbcc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.817656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1190/2419] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:13.821378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.832126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10626e33-bc0e-4e23-9a41-89ee17afdf31 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:13.844477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-24b3b824-e15e-48ad-a26c-9fa49fcbcc97 None None] MFA Rules not processed for user `3cb2514bd57747a59a19ac2e2d392670`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:13.845736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-24b3b824-e15e-48ad-a26c-9fa49fcbcc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:13.849949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:13.869661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:13.897649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:13.898537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:13.898820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:13.899095 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] PATH_INFO: `/v3/projects/dd70afa71f6341978f6c016fc5311e38/users/080cfbd9b11144fbbe7a9830068d3189/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:13.914818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd70afa71f6341978f6c016fc5311e38, user_id=080cfbd9b11144fbbe7a9830068d3189, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:13.917627 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:13.922730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1230/2420] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:13.927555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3b824-e15e-48ad-a26c-9fa49fcbcc97 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.931272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1191/2421] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:13 2022] PUT /identity/v3/projects/dd70afa71f6341978f6c016fc5311e38/users/080cfbd9b11144fbbe7a9830068d3189/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:13.934975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:13.938897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3b824-e15e-48ad-a26c-9fa49fcbcc97 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.010616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8cfbc-accc-4914-b6cb-ba1af87cb86c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.015412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.016077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.016365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.016477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.016695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.017083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.017333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.017441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.017624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.018377 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.019223 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.019944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.033894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1192/2422] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:14.040639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.052860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89dc14-0939-4251-bce3-78b8312016cc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.053053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1231/2423] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:13 2022] POST /identity/v3/projects => generated 357 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:14.057288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.124813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc383e-562e-448e-8786-5b78ed3908bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.125963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc383e-562e-448e-8786-5b78ed3908bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.126247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc383e-562e-448e-8786-5b78ed3908bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.126521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc383e-562e-448e-8786-5b78ed3908bc admin admin] PATH_INFO: `/v3/projects/dd70afa71f6341978f6c016fc5311e38/users/080cfbd9b11144fbbe7a9830068d3189/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.128653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc383e-562e-448e-8786-5b78ed3908bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd70afa71f6341978f6c016fc5311e38, user_id=080cfbd9b11144fbbe7a9830068d3189, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.131332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc383e-562e-448e-8786-5b78ed3908bc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.133426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef280-7791-4b66-be8b-480117decd20 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.139926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.140824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.140920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.141184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.141892 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.144499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.149602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1193/2424] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:14 2022] PUT /identity/v3/projects/dd70afa71f6341978f6c016fc5311e38/users/080cfbd9b11144fbbe7a9830068d3189/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:14.164453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.164899 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.165197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.191466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1232/2425] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:14 2022] POST /identity/v3/users => generated 285 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:14.195241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.203075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] MFA Rules not processed for user `080cfbd9b11144fbbe7a9830068d3189`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:14.204274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:14.213598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c164-2aba-43f8-8fad-445618fd4af1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.230963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:14.250594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:14.283331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.283727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1194/2426] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:14.284000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.284271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.284580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.285301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.286914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.296634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.298496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:14.300719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1233/2427] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:14.306133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.316762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b6719-da70-4755-9ebf-39c26fd1c949 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.322707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.323588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.323855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.324109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] PATH_INFO: `/v3/projects/c1b65ea8d65945e7afbcbef2381fd416/users/45b165d521c74d77898607be65f685c2/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.340835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b65ea8d65945e7afbcbef2381fd416, user_id=45b165d521c74d77898607be65f685c2, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.343684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.358682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1234/2428] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:14 2022] PUT /identity/v3/projects/c1b65ea8d65945e7afbcbef2381fd416/users/45b165d521c74d77898607be65f685c2/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:14.363009 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.381339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165b465-fa85-4e40-ac9d-2771b6ccad0e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.445816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6e697-851d-46fe-bf99-88af5b741279 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.448598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.449514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.449987 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.450544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.451467 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.455691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.456237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.456718 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.457748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.459582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.470690 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1235/2429] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:14.474571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.486301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1281c0c-d685-47c1-8fc7-66a91a474f8c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.491619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c665bba-0f78-4c65-b8b1-a114936fddaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.492574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c665bba-0f78-4c65-b8b1-a114936fddaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.496598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c665bba-0f78-4c65-b8b1-a114936fddaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.497479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c665bba-0f78-4c65-b8b1-a114936fddaf admin admin] PATH_INFO: `/v3/projects/c1b65ea8d65945e7afbcbef2381fd416/users/45b165d521c74d77898607be65f685c2/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.499526 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c665bba-0f78-4c65-b8b1-a114936fddaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b65ea8d65945e7afbcbef2381fd416, user_id=45b165d521c74d77898607be65f685c2, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.502352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c665bba-0f78-4c65-b8b1-a114936fddaf admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.513394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1236/2430] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:14 2022] PUT /identity/v3/projects/c1b65ea8d65945e7afbcbef2381fd416/users/45b165d521c74d77898607be65f685c2/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:14.524848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.525132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.525493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.533966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.535577 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.546931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1195/2431] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:14.559855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] MFA Rules not processed for user `45b165d521c74d77898607be65f685c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:14.561100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:14.644496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.656333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e075c0d-91df-4f28-8f41-e0e2f7ffbd44 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.656903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:14.664562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1237/2432] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:14.674172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.677290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:14.704797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:14.716936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.717653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.718555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.718555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.753017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eabfda-661e-4773-9a3b-6801d2431bbc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.756790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.757592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.757922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.758228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.769317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.770764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.781395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1238/2433] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:14.799995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.801906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.813807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1196/2434] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:14.864268 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:14.873227 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8ac2c-8da9-4343-ad99-d120215e08fe service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:14.878062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:14.879094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:14.879559 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:14.880094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:14.964232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:14.965715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:14.980187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1239/2435] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:19.734855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:19.800409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:19.801861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:19.814717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb35be3-ee59-41c9-a9e6-7d0a1e71e3fd service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:19.819534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:19.820442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:19.820725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:19.820998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] PATH_INFO: `/v3/users/36bbee8799084903bf70ccb387a0244d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:19.822207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bbee8799084903bf70ccb387a0244d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:19.824194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:19.860455 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a109431-de38-404f-b454-ca29fe19da25 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:19.864138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57777fb3-779f-403d-90c5-cd514b5d964a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:19.864810 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777fb3-779f-403d-90c5-cd514b5d964a service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:19.865088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777fb3-779f-403d-90c5-cd514b5d964a service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:19.865362 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777fb3-779f-403d-90c5-cd514b5d964a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:19.867119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-57777fb3-779f-403d-90c5-cd514b5d964a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:19.875281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bbee8799084903bf70ccb387a0244d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:19.875937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bbee8799084903bf70ccb387a0244d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:19.879719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57777fb3-779f-403d-90c5-cd514b5d964a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:19.881248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57777fb3-779f-403d-90c5-cd514b5d964a service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:19.883110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bbee8799084903bf70ccb387a0244d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:19.883879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bbee8799084903bf70ccb387a0244d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:19.892313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bbee8799084903bf70ccb387a0244d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:19.892740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1240/2436] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:19.910318 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1197/2437] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:19 2022] DELETE /identity/v3/users/36bbee8799084903bf70ccb387a0244d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:19.914794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:19.915083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:19.915409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:19.932558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:19.933782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:19.980395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:20.006689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1241/2438] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:20.017435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:20.018881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:20.034309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:20.080314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578996d-6f34-4b1d-a2b8-d820d3e8fa8e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:20.085901 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:20.086582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:20.086865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:20.087143 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:20.097404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:20.098838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:20.109552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1198/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:20.534464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:20.536001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:20.559026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5ee77-49f0-429f-9f60-9619af8215fe None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:20.570049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:20.570986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:20.571217 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:20.571618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] PATH_INFO: `/v3/projects/d879eaf8a03f45c2b1d8b5f925f80901` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:20.573031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d879eaf8a03f45c2b1d8b5f925f80901)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:20.574702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:20.634158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd879eaf8a03f45c2b1d8b5f925f80901', 'request_id': 'req-40cce866-f775-4fc7-a1a4-be282e32da77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.640664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd879eaf8a03f45c2b1d8b5f925f80901', 'request_id': 'req-40cce866-f775-4fc7-a1a4-be282e32da77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.647521 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd879eaf8a03f45c2b1d8b5f925f80901', 'request_id': 'req-40cce866-f775-4fc7-a1a4-be282e32da77'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.662083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] The token cache is being invalidate because project d879eaf8a03f45c2b1d8b5f925f80901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:20.662378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.663032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.664704 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1242/2440] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:20 2022] DELETE /identity/v3/projects/d879eaf8a03f45c2b1d8b5f925f80901 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:20.668607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:20.678599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881121c8-6efc-4e5a-9a82-708bffe8f017 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:20.739587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:20.740494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:20.740775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:20.741047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] PATH_INFO: `/v3/users/080cfbd9b11144fbbe7a9830068d3189` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:20.742210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] RBAC: Authorizing `identity:delete_user(user_id=080cfbd9b11144fbbe7a9830068d3189)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:20.744125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:20.794257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080cfbd9b11144fbbe7a9830068d3189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.794933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080cfbd9b11144fbbe7a9830068d3189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.803084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080cfbd9b11144fbbe7a9830068d3189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.803851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080cfbd9b11144fbbe7a9830068d3189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.811442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-327806d7-5cb7-42ae-959a-0ae91e0e55de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080cfbd9b11144fbbe7a9830068d3189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:20.830220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1199/2441] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:20 2022] DELETE /identity/v3/users/080cfbd9b11144fbbe7a9830068d3189 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:21.142497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:21.144673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:21.160298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce866-f775-4fc7-a1a4-be282e32da77 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:21.228100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:21.229380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.229380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.229602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] PATH_INFO: `/v3/projects/dd70afa71f6341978f6c016fc5311e38` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.230787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd70afa71f6341978f6c016fc5311e38)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:21.232990 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:21.282442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd70afa71f6341978f6c016fc5311e38', 'request_id': 'req-6ad13638-8d5d-47cf-a814-0143200c7f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:21.288707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd70afa71f6341978f6c016fc5311e38', 'request_id': 'req-6ad13638-8d5d-47cf-a814-0143200c7f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:21.295041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd70afa71f6341978f6c016fc5311e38', 'request_id': 'req-6ad13638-8d5d-47cf-a814-0143200c7f17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:21.304473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:21.310848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] The token cache is being invalidate because project dd70afa71f6341978f6c016fc5311e38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:21.311137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:21.311956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6ad13638-8d5d-47cf-a814-0143200c7f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:21.317656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1243/2442] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:21 2022] DELETE /identity/v3/projects/dd70afa71f6341978f6c016fc5311e38 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:21.320192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e4e11-2835-480e-9cec-596fe1705cef None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.320516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e4e11-2835-480e-9cec-596fe1705cef None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.320803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e4e11-2835-480e-9cec-596fe1705cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.340588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-242e4e11-2835-480e-9cec-596fe1705cef None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:21.341796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-242e4e11-2835-480e-9cec-596fe1705cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:21.389905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:21.430879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1200/2443] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:21.438615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b563-17f3-4958-b330-7a5426fd18e0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.438981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b563-17f3-4958-b330-7a5426fd18e0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.440450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b563-17f3-4958-b330-7a5426fd18e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.458531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2419b563-17f3-4958-b330-7a5426fd18e0 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:21.459768 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2419b563-17f3-4958-b330-7a5426fd18e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:21.469054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1244/2444] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:21.473180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e4e11-2835-480e-9cec-596fe1705cef None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:21.485486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e4e11-2835-480e-9cec-596fe1705cef None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:21.490663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:21.491551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.491816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.493396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.493396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:21.493396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:21.495856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:21.512755 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1201/2445] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:21.514018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419b563-17f3-4958-b330-7a5426fd18e0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:21.524371 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419b563-17f3-4958-b330-7a5426fd18e0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:21.530200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:21.531440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.531440 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.531717 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.532537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:21.534535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:21.568778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1245/2446] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:21 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:21.570725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:21.592520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce723-0152-4b8f-a120-c396af2f8938 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:21.725273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:21.726047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.726820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.726820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.727316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:21.729284 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:21.766495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1202/2447] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:21 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:21.767303 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:21.779225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa48c9-58de-4038-90f5-5a84be59e731 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:21.785143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:21.785965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.786324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.786672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.787554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:21.789365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:21.804409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1246/2448] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:21.808181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:21.823780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be59c7-b38e-47c8-a330-29093941b9a2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:21.828955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3640e-9a33-49be-b987-fb732b050f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:21.829630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3640e-9a33-49be-b987-fb732b050f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.832210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3640e-9a33-49be-b987-fb732b050f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.832547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3640e-9a33-49be-b987-fb732b050f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.833305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3640e-9a33-49be-b987-fb732b050f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:21.835087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3640e-9a33-49be-b987-fb732b050f52 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:21.846768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1203/2449] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:21.851075 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:21.873094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d498-c895-484c-b45b-c362785478ab admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:21.878687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:21.879571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.879853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.880131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] PATH_INFO: `/v3/projects/816567bd76d84e5f9de58969bf4ebc2c/users/b4425a2ff4174303a24d9a816d70e49a/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.901350 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=816567bd76d84e5f9de58969bf4ebc2c, user_id=b4425a2ff4174303a24d9a816d70e49a, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:21.904809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:21.926520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1247/2450] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:21 2022] PUT /identity/v3/projects/816567bd76d84e5f9de58969bf4ebc2c/users/b4425a2ff4174303a24d9a816d70e49a/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:21.931662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bfe43-a39a-45d9-a83f-91cef08a5446 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:21.932000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bfe43-a39a-45d9-a83f-91cef08a5446 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:21.932348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bfe43-a39a-45d9-a83f-91cef08a5446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:21.969793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ef5bfe43-a39a-45d9-a83f-91cef08a5446 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:21.972257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef5bfe43-a39a-45d9-a83f-91cef08a5446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:22.062483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1204/2451] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:22.072766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:22.074523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:22.155215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd343ad-cf9b-461c-88c7-c16190ae6512 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:22.160305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:22.161026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:22.161367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:22.161538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:22.171921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:22.173403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:22.184194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1248/2452] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:23.213895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581cb8c-1b21-43e2-af3e-bec73825c653 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:23.214956 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581cb8c-1b21-43e2-af3e-bec73825c653 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:23.215450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581cb8c-1b21-43e2-af3e-bec73825c653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:23.257767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4581cb8c-1b21-43e2-af3e-bec73825c653 None None] MFA Rules not processed for user `b4425a2ff4174303a24d9a816d70e49a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:23.259182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4581cb8c-1b21-43e2-af3e-bec73825c653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:23.307355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:23.388021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1205/2453] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:23.391629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:23.402191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc41179-fb26-4a35-9224-23e7f384ec74 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:23.416584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:23.417271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:23.417552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:23.417824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:23.418523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:23.420474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:23.473610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1249/2454] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:23 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:23.477509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581cb8c-1b21-43e2-af3e-bec73825c653 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:23.493880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581cb8c-1b21-43e2-af3e-bec73825c653 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:23.594891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:23.595591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:23.595881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:23.596173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:23.596908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:23.598866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:23.625322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:23.627116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:23.641965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1206/2455] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:23 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:23.644872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:23.653554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714ec79-dde2-415b-8eea-21020185fe24 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:23.666575 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ff578-0534-4637-b349-6e0c4558f0b4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:23.669474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:23.670546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:23.671035 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:23.671333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] PATH_INFO: `/v3/users/213a7f6737504f9eb89a2e51ddd639ff` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:23.672680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213a7f6737504f9eb89a2e51ddd639ff)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:23.674588 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:23.680579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:23.681362 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:23.681653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:23.681939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:23.682727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:23.684747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:23.699380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1207/2456] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:23.705371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:23.726872 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e68979-a3ae-470a-80f3-9d499f898eed admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:23.737547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:23.738781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:23.739208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:23.739524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] PATH_INFO: `/v3/projects/7f92aa0b9b6d413aa436a2f81fa9eeff/users/a3619c1469a64861af3d1dde7ff4ed24/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:23.783267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213a7f6737504f9eb89a2e51ddd639ff'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:23.784412 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213a7f6737504f9eb89a2e51ddd639ff'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:23.785231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213a7f6737504f9eb89a2e51ddd639ff'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:23.789929 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f92aa0b9b6d413aa436a2f81fa9eeff, user_id=a3619c1469a64861af3d1dde7ff4ed24, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:23.796768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:23.797666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213a7f6737504f9eb89a2e51ddd639ff'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:23.810639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:23.813475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213a7f6737504f9eb89a2e51ddd639ff'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:23.829067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1208/2457] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:23 2022] PUT /identity/v3/projects/7f92aa0b9b6d413aa436a2f81fa9eeff/users/a3619c1469a64861af3d1dde7ff4ed24/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:23.836989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:23.837447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1250/2458] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:23 2022] DELETE /identity/v3/users/213a7f6737504f9eb89a2e51ddd639ff => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:23.837648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:23.918998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee1ea0-3e24-470b-a68d-40ac6efb4895 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:23.949836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:23.980653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:23.981322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:23.981858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:23.982137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:23.982845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:23.984876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:23.995302 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1209/2459] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:23.999643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.010148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061693ba-bbc5-4dc8-a0b3-8abd676ae806 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.015473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.016649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.016953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.017238 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] PATH_INFO: `/v3/projects/7f92aa0b9b6d413aa436a2f81fa9eeff/users/a3619c1469a64861af3d1dde7ff4ed24/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.019355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f92aa0b9b6d413aa436a2f81fa9eeff, user_id=a3619c1469a64861af3d1dde7ff4ed24, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.022064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.027843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.030161 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:24.034077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1251/2460] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:23 2022] PUT /identity/v3/projects/7f92aa0b9b6d413aa436a2f81fa9eeff/users/a3619c1469a64861af3d1dde7ff4ed24/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:24.047659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47076802-7e12-4fee-ab22-610189ef9ea0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.113009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.114054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.114334 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.114604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] PATH_INFO: `/v3/projects/d9f6ff4cee09476c925730a31d32e56f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.116024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f6ff4cee09476c925730a31d32e56f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.118131 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.124264 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.126191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:24.198254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f6ff4cee09476c925730a31d32e56f', 'request_id': 'req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:24.208174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f6ff4cee09476c925730a31d32e56f', 'request_id': 'req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:24.208547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16caf5c5-6053-4cba-a43e-836ebe008a37 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.217364 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f6ff4cee09476c925730a31d32e56f', 'request_id': 'req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:24.234718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] The token cache is being invalidate because project d9f6ff4cee09476c925730a31d32e56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:24.235081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:24.236582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8a7c5aba-2ba0-410c-8f87-bb6bd3d7dddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:24.241918 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1210/2461] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:24 2022] DELETE /identity/v3/projects/d9f6ff4cee09476c925730a31d32e56f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:24.243280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122319d5-7d3c-44f3-9371-81ae15a3e899 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.243569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122319d5-7d3c-44f3-9371-81ae15a3e899 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.243844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122319d5-7d3c-44f3-9371-81ae15a3e899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.262530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-122319d5-7d3c-44f3-9371-81ae15a3e899 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:24.262963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.263725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-122319d5-7d3c-44f3-9371-81ae15a3e899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:24.264285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.265488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.265917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.267885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:24.323781 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:24.338051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1211/2462] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:24.345950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc4a95-5854-4d71-b02c-4b5c80d9170a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.346438 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc4a95-5854-4d71-b02c-4b5c80d9170a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.346784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc4a95-5854-4d71-b02c-4b5c80d9170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.349432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.351276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.364146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1252/2463] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:24.371224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b4bc4a95-5854-4d71-b02c-4b5c80d9170a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:24.372500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b4bc4a95-5854-4d71-b02c-4b5c80d9170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:24.386084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1212/2464] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:24.387805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.401306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577481e-8eaa-4039-9be5-47fa7b860cc0 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.408887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.409647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.410051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.410399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.411194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:24.411779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.414269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.429055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1253/2465] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:24.433243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc4a95-5854-4d71-b02c-4b5c80d9170a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.444389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc4a95-5854-4d71-b02c-4b5c80d9170a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.449807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.450410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.450672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.450965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.451802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.454452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.496628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1213/2466] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:24 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:24.498871 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.512792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712a85a-2ea7-49b2-919a-a208dfaaaff5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.556993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:24.600623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.601459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.601982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.602493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.603407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.605671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.639508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.648671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1254/2467] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:24 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:24.652350 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cae52f-6441-4889-87c8-d617f7a11881 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.659276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.659997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.660279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.660589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.661352 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.663142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.680409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1214/2468] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:24.682918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.694920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deefc9-62a9-4865-b0f5-ad696473f0ef admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.699563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d3c26-0f54-46d5-b7d5-582c9a8ebab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.700232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d3c26-0f54-46d5-b7d5-582c9a8ebab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.700536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d3c26-0f54-46d5-b7d5-582c9a8ebab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.700811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d3c26-0f54-46d5-b7d5-582c9a8ebab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.701530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d3c26-0f54-46d5-b7d5-582c9a8ebab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.703905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d3c26-0f54-46d5-b7d5-582c9a8ebab8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.714096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1255/2469] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:24.718908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:24.729205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1b2f-c0b1-4ae6-b5cf-84cb0cf27479 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:24.734847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67d460-594a-4a89-bbe8-878a1ef4b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:24.735878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67d460-594a-4a89-bbe8-878a1ef4b60b admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.736157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67d460-594a-4a89-bbe8-878a1ef4b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.736450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67d460-594a-4a89-bbe8-878a1ef4b60b admin admin] PATH_INFO: `/v3/projects/1e9fa7cbfb084980ab4bb283a43e4cc9/users/9407d656b1be4155afa7d921bd2f41e7/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.751722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67d460-594a-4a89-bbe8-878a1ef4b60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9fa7cbfb084980ab4bb283a43e4cc9, user_id=9407d656b1be4155afa7d921bd2f41e7, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:24.754312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67d460-594a-4a89-bbe8-878a1ef4b60b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:24.770255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1215/2470] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:24 2022] PUT /identity/v3/projects/1e9fa7cbfb084980ab4bb283a43e4cc9/users/9407d656b1be4155afa7d921bd2f41e7/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:24.780318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491cc8d-e20a-4507-84c8-50f0f727dce7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.781141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491cc8d-e20a-4507-84c8-50f0f727dce7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.781629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491cc8d-e20a-4507-84c8-50f0f727dce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.832002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cef27-49a2-438d-bdff-71bded817090 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:24.832513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cef27-49a2-438d-bdff-71bded817090 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:24.832893 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cef27-49a2-438d-bdff-71bded817090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:24.838883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7491cc8d-e20a-4507-84c8-50f0f727dce7 None None] MFA Rules not processed for user `9407d656b1be4155afa7d921bd2f41e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:24.840112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7491cc8d-e20a-4507-84c8-50f0f727dce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:24.873844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-508cef27-49a2-438d-bdff-71bded817090 None None] MFA Rules not processed for user `a3619c1469a64861af3d1dde7ff4ed24`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:24.882280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-508cef27-49a2-438d-bdff-71bded817090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:24.950482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1256/2471] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:24.957518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491cc8d-e20a-4507-84c8-50f0f727dce7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.013841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1216/2472] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:25.068886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cef27-49a2-438d-bdff-71bded817090 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.070281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-508cef27-49a2-438d-bdff-71bded817090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:25.076779 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491cc8d-e20a-4507-84c8-50f0f727dce7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.082310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.083157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.083425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.083697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.084410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.086470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.119434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1257/2473] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:24 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:25.123527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.146375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cef27-49a2-438d-bdff-71bded817090 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.160434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:25.189981 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:25.210070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.210719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.211084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4a0a3-9393-4a6c-bd19-a99a7803f619 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.211203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.211421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.213524 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:25.216734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.217488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.218248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.218587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.219276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.221595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.246989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1258/2474] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:25 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:25.252932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.263476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422c29f-2146-4cda-9783-44cfd490e0f4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.268583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.269320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.269678 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.270020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.270818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.272506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.284784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.285761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1259/2475] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:25.286707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.289337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.298134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1217/2476] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:25.312525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604c5cd-452d-4cc1-af14-34059841c7e6 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.318189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.319302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.319758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.320191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] PATH_INFO: `/v3/projects/9de152d4cef14873988907fe863b04df/users/10fd69b486714231bbed75e5cc47e8aa/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.342566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de152d4cef14873988907fe863b04df, user_id=10fd69b486714231bbed75e5cc47e8aa, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.347101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.365257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1260/2477] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:25 2022] PUT /identity/v3/projects/9de152d4cef14873988907fe863b04df/users/10fd69b486714231bbed75e5cc47e8aa/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:25.369136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.385542 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332ba4b-6c56-4820-a59e-d1777b64b080 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.473069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee821492-b1a2-4baf-a250-d37ef727bb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.474150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee821492-b1a2-4baf-a250-d37ef727bb3e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.474518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee821492-b1a2-4baf-a250-d37ef727bb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.475093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee821492-b1a2-4baf-a250-d37ef727bb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.476384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee821492-b1a2-4baf-a250-d37ef727bb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.478808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee821492-b1a2-4baf-a250-d37ef727bb3e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.490585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1218/2478] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:25.495241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.507208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9421d-151f-433e-b9e4-7a71a58cebca admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.512446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.513883 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.514448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.515577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] PATH_INFO: `/v3/projects/9de152d4cef14873988907fe863b04df/users/10fd69b486714231bbed75e5cc47e8aa/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.519993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de152d4cef14873988907fe863b04df, user_id=10fd69b486714231bbed75e5cc47e8aa, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.522750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.535794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1261/2479] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:25 2022] PUT /identity/v3/projects/9de152d4cef14873988907fe863b04df/users/10fd69b486714231bbed75e5cc47e8aa/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:25.548241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef737-33bb-45cf-872c-f86e8e8c879b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.548557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef737-33bb-45cf-872c-f86e8e8c879b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.548834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef737-33bb-45cf-872c-f86e8e8c879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.584914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-219ef737-33bb-45cf-872c-f86e8e8c879b None None] MFA Rules not processed for user `10fd69b486714231bbed75e5cc47e8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:25.586310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-219ef737-33bb-45cf-872c-f86e8e8c879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:25.675124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1219/2480] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:25.689305 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:25.691398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:25.758657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a1661-0bf8-4819-a5f0-851d5985e9a8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:25.762241 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de206ce-7825-43be-9944-8fab9b128fdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:25.762878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de206ce-7825-43be-9944-8fab9b128fdc service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:25.763157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de206ce-7825-43be-9944-8fab9b128fdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:25.763432 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de206ce-7825-43be-9944-8fab9b128fdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:25.773270 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de206ce-7825-43be-9944-8fab9b128fdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:25.774684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de206ce-7825-43be-9944-8fab9b128fdc service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:25.784175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1262/2481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:26.163021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef737-33bb-45cf-872c-f86e8e8c879b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:26.177734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:26.226480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef737-33bb-45cf-872c-f86e8e8c879b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:26.231060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:26.232208 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:26.232521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:26.232804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] PATH_INFO: `/v3/users/9407d656b1be4155afa7d921bd2f41e7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:26.234494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9407d656b1be4155afa7d921bd2f41e7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:26.236428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:26.281508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9407d656b1be4155afa7d921bd2f41e7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.282072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9407d656b1be4155afa7d921bd2f41e7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.288921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9407d656b1be4155afa7d921bd2f41e7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.289552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9407d656b1be4155afa7d921bd2f41e7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.297544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9407d656b1be4155afa7d921bd2f41e7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.315341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1220/2482] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:26 2022] DELETE /identity/v3/users/9407d656b1be4155afa7d921bd2f41e7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:26.319838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:26.320130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:26.320447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:26.338337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:26.339524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:26.399363 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1263/2483] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:26.409576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:26.410999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:26.467498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f3033-b43e-4153-ab7b-df01f2914272 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:26.471769 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:26.472454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:26.472741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:26.473022 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:26.482202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:26.484466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:26.493980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1221/2484] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:26.764342 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:26.765858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:26.777909 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d07e4d-9336-40aa-b62e-fc55ac91516d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:26.783114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:26.783977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:26.784257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:26.784565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] PATH_INFO: `/v3/projects/1e9fa7cbfb084980ab4bb283a43e4cc9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:26.785752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9fa7cbfb084980ab4bb283a43e4cc9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:26.787961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:26.790716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:26.838494 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9fa7cbfb084980ab4bb283a43e4cc9', 'request_id': 'req-387431c8-5ca5-4545-bea7-528a00f4e437'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.844785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9fa7cbfb084980ab4bb283a43e4cc9', 'request_id': 'req-387431c8-5ca5-4545-bea7-528a00f4e437'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.850931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9fa7cbfb084980ab4bb283a43e4cc9', 'request_id': 'req-387431c8-5ca5-4545-bea7-528a00f4e437'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.864282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] The token cache is being invalidate because project 1e9fa7cbfb084980ab4bb283a43e4cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:26.864597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.865213 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.866745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1264/2485] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:26 2022] DELETE /identity/v3/projects/1e9fa7cbfb084980ab4bb283a43e4cc9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:26.870833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:26.880324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41544c-0009-4c77-ad88-5c252f6856e9 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:26.913543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:26.939276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:26.940148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:26.940463 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:26.940743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] PATH_INFO: `/v3/users/10fd69b486714231bbed75e5cc47e8aa` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:26.941909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10fd69b486714231bbed75e5cc47e8aa)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:26.943818 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:26.989217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10fd69b486714231bbed75e5cc47e8aa'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.989880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10fd69b486714231bbed75e5cc47e8aa'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.996914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10fd69b486714231bbed75e5cc47e8aa'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:26.997531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10fd69b486714231bbed75e5cc47e8aa'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:27.004254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-132de0e3-ce4b-49bc-873f-dd780e996168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10fd69b486714231bbed75e5cc47e8aa'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:27.021757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1222/2486] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:26 2022] DELETE /identity/v3/users/10fd69b486714231bbed75e5cc47e8aa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:27.299459 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:27.300947 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:27.316968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387431c8-5ca5-4545-bea7-528a00f4e437 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:27.376156 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:27.377063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.377374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.377656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] PATH_INFO: `/v3/projects/9de152d4cef14873988907fe863b04df` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.378875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de152d4cef14873988907fe863b04df)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:27.381179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:27.427329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de152d4cef14873988907fe863b04df', 'request_id': 'req-66c4130b-f47b-4078-b91d-73241d0323fa'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:27.432887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de152d4cef14873988907fe863b04df', 'request_id': 'req-66c4130b-f47b-4078-b91d-73241d0323fa'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:27.439563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de152d4cef14873988907fe863b04df', 'request_id': 'req-66c4130b-f47b-4078-b91d-73241d0323fa'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:27.452800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] The token cache is being invalidate because project 9de152d4cef14873988907fe863b04df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:27.453162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:27.453797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-66c4130b-f47b-4078-b91d-73241d0323fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:27.455460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1265/2487] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:27 2022] DELETE /identity/v3/projects/9de152d4cef14873988907fe863b04df => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:27.460786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c8a4-bc81-4551-bebd-e8d35eff0c77 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.461243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c8a4-bc81-4551-bebd-e8d35eff0c77 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.461789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c8a4-bc81-4551-bebd-e8d35eff0c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.480952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f6e7c8a4-bc81-4551-bebd-e8d35eff0c77 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:27.482403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f6e7c8a4-bc81-4551-bebd-e8d35eff0c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:27.542857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1223/2488] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:27.551697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7563b4-6feb-4eca-8761-3fb96fb2a23f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.551921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7563b4-6feb-4eca-8761-3fb96fb2a23f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.552231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7563b4-6feb-4eca-8761-3fb96fb2a23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.572558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-8d7563b4-6feb-4eca-8761-3fb96fb2a23f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:27.573762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8d7563b4-6feb-4eca-8761-3fb96fb2a23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:27.582000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1266/2489] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:27.586277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7c8a4-bc81-4551-bebd-e8d35eff0c77 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:27.598127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7c8a4-bc81-4551-bebd-e8d35eff0c77 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:27.603183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:27.603939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.604225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.604547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.605278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:27.605586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:27.607454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:27.623170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1224/2490] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:27.625024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7563b4-6feb-4eca-8761-3fb96fb2a23f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:27.634564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7563b4-6feb-4eca-8761-3fb96fb2a23f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:27.639473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:27.640194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.640586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.640879 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.641592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:27.643501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:27.657744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:27.673252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1267/2491] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:27 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:27.677495 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:27.688605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da877eb6-de1a-4a40-b976-31535c571a60 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:27.730460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:27.755380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:27.756322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.756988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.757360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.758128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:27.760289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:27.785342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1225/2492] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:27 2022] POST /identity/v3/users => generated 282 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:27.790167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:27.804644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8f5d-754c-4f3c-8c91-db92381a897e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:27.809450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:27.810199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.810578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.810916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.812618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:27.814513 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:27.825862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1268/2493] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:27.830409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:27.841202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa789f-9af2-4f17-86d5-4494725a1907 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:27.846485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38925ba-ab07-40ad-ae22-016352698304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:27.847156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38925ba-ab07-40ad-ae22-016352698304 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.847418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38925ba-ab07-40ad-ae22-016352698304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.847691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38925ba-ab07-40ad-ae22-016352698304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.848475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38925ba-ab07-40ad-ae22-016352698304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:27.850061 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38925ba-ab07-40ad-ae22-016352698304 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:27.862475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1226/2494] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:27.866953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:27.878557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5af95-949f-4ae8-a021-2877a90d8b1f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:27.883192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:27.884084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.884372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.884667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] PATH_INFO: `/v3/projects/aaad9a77b56043de8b38450ec47ecb0e/users/dcd51c4ab0a54afc8a60c83aac101fe3/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.899996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaad9a77b56043de8b38450ec47ecb0e, user_id=dcd51c4ab0a54afc8a60c83aac101fe3, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:27.902844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:27.916580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1269/2495] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:27 2022] PUT /identity/v3/projects/aaad9a77b56043de8b38450ec47ecb0e/users/dcd51c4ab0a54afc8a60c83aac101fe3/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:27.932011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955824-749c-419c-9f27-e72869d19ee1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:27.932566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955824-749c-419c-9f27-e72869d19ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:27.933056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955824-749c-419c-9f27-e72869d19ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:27.964196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-32955824-749c-419c-9f27-e72869d19ee1 None None] MFA Rules not processed for user `dcd51c4ab0a54afc8a60c83aac101fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:27.966213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-32955824-749c-419c-9f27-e72869d19ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:28.039509 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1227/2496] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:28.044899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.123679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb690c4e-fdb9-43c7-9c61-e2cffc0973cb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.129357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.130248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.130718 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.131421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.132282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.134763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.179367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1270/2497] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:28 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:28.183858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32955824-749c-419c-9f27-e72869d19ee1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.197124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32955824-749c-419c-9f27-e72869d19ee1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.258193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.258866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.259177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.259456 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.260138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.262082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.282667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1228/2498] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:28 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:28.286866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.297739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451acacb-e9cd-4363-9a99-c78f639e0805 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.302681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.303333 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.303616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.303891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.304714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.306302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.316999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1271/2499] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:28.321218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.331635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71d77-b360-4ecd-8bfd-62b01cad3ed4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.336053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.336954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.337231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.337502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] PATH_INFO: `/v3/projects/dd0b759e78d54419800ec141a9dfdc68/users/f1b84fb86b674c7ab0bb4091a4685ab5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.352622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0b759e78d54419800ec141a9dfdc68, user_id=f1b84fb86b674c7ab0bb4091a4685ab5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.355295 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.368648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1229/2500] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:28 2022] PUT /identity/v3/projects/dd0b759e78d54419800ec141a9dfdc68/users/f1b84fb86b674c7ab0bb4091a4685ab5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:28.372499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.382821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab433b-fe21-4640-892f-2c831e984beb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.440068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63624a29-b338-45a9-bfdd-a50c9d2d3128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.440772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63624a29-b338-45a9-bfdd-a50c9d2d3128 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.441056 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63624a29-b338-45a9-bfdd-a50c9d2d3128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.441337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63624a29-b338-45a9-bfdd-a50c9d2d3128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.442057 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63624a29-b338-45a9-bfdd-a50c9d2d3128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.443634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63624a29-b338-45a9-bfdd-a50c9d2d3128 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.453792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1272/2501] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:28.457983 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.468754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a0941-8969-4f0f-9675-5623f548f126 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.473420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.474316 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.474593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.474866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] PATH_INFO: `/v3/projects/dd0b759e78d54419800ec141a9dfdc68/users/f1b84fb86b674c7ab0bb4091a4685ab5/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.476822 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0b759e78d54419800ec141a9dfdc68, user_id=f1b84fb86b674c7ab0bb4091a4685ab5, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.479419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.490141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1230/2502] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:28 2022] PUT /identity/v3/projects/dd0b759e78d54419800ec141a9dfdc68/users/f1b84fb86b674c7ab0bb4091a4685ab5/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:28.501285 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213ecdf-b6f5-487c-acc2-9e32ae113463 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.501571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213ecdf-b6f5-487c-acc2-9e32ae113463 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.501849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213ecdf-b6f5-487c-acc2-9e32ae113463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.529174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2213ecdf-b6f5-487c-acc2-9e32ae113463 None None] MFA Rules not processed for user `f1b84fb86b674c7ab0bb4091a4685ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:28.531309 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2213ecdf-b6f5-487c-acc2-9e32ae113463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:28.547834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:28.600427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1273/2503] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:28.611979 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.613674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:28.618910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:28.681729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ec891-2bee-48c6-bd2b-e1179bebbe4c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.685930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.687989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.688511 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.688664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.698222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.699740 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.711329 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1231/2504] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:28.786693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213ecdf-b6f5-487c-acc2-9e32ae113463 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.795872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213ecdf-b6f5-487c-acc2-9e32ae113463 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.800600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.801402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.801692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.801966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.841949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.844891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:28.874622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.876051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.890952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1274/2505] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:28.898648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:28.900335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:28.913041 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65792db8-bd36-46e8-8a6f-90c290399c22 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.918525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb3833-7f45-4dca-86a1-6af97c53afa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.919162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb3833-7f45-4dca-86a1-6af97c53afa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.919439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb3833-7f45-4dca-86a1-6af97c53afa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.919709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb3833-7f45-4dca-86a1-6af97c53afa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.920201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd25a-a766-49f6-975f-06077f30e626 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:28.922497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7aeb3833-7f45-4dca-86a1-6af97c53afa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:28.925623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:28.926863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:28.927246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:28.927631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] PATH_INFO: `/v3/users/9bc58b7d56b340119a39ea7d86ee30ff` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:28.929033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc58b7d56b340119a39ea7d86ee30ff)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.931543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:28.985539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc58b7d56b340119a39ea7d86ee30ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:28.986322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc58b7d56b340119a39ea7d86ee30ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:28.994117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc58b7d56b340119a39ea7d86ee30ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:28.995651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc58b7d56b340119a39ea7d86ee30ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:28.997545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb3833-7f45-4dca-86a1-6af97c53afa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:28.999568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb3833-7f45-4dca-86a1-6af97c53afa6 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.011601 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1275/2506] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:28 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:29.018294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc58b7d56b340119a39ea7d86ee30ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:29.043048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1232/2507] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:28 2022] DELETE /identity/v3/users/9bc58b7d56b340119a39ea7d86ee30ff => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:29.048524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.050419 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.050708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.068743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:29.070160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:29.115599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:29.136004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1276/2508] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:29.145514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:29.147075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:29.177565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:29.219320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276374a-33d1-49b7-ad8b-76743daacf54 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:29.223356 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447231f-18ce-446f-bcd7-d129fe29e456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:29.224020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447231f-18ce-446f-bcd7-d129fe29e456 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.224294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447231f-18ce-446f-bcd7-d129fe29e456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.224601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447231f-18ce-446f-bcd7-d129fe29e456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.234898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447231f-18ce-446f-bcd7-d129fe29e456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:29.236677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447231f-18ce-446f-bcd7-d129fe29e456 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.255678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1233/2509] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:29.518677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:29.520091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:29.534633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93761c9-ee37-4283-b1c3-a444d8d063e1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:29.539373 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:29.540235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.540549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.540818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] PATH_INFO: `/v3/projects/29fcd5de091e4a82ac71d8085be7144e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.541982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29fcd5de091e4a82ac71d8085be7144e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:29.544024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.590797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29fcd5de091e4a82ac71d8085be7144e', 'request_id': 'req-344acddc-bf08-43b3-96b8-c243ca7f2eb7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:29.596516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29fcd5de091e4a82ac71d8085be7144e', 'request_id': 'req-344acddc-bf08-43b3-96b8-c243ca7f2eb7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:29.603375 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29fcd5de091e4a82ac71d8085be7144e', 'request_id': 'req-344acddc-bf08-43b3-96b8-c243ca7f2eb7'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:29.615753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] The token cache is being invalidate because project 29fcd5de091e4a82ac71d8085be7144e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:29.616042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:29.616662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-344acddc-bf08-43b3-96b8-c243ca7f2eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:29.618188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1277/2510] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:29 2022] DELETE /identity/v3/projects/29fcd5de091e4a82ac71d8085be7144e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:29.623112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037c060-6bd1-4aab-909e-d399d08481d5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.623574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037c060-6bd1-4aab-909e-d399d08481d5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.623670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037c060-6bd1-4aab-909e-d399d08481d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.642836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d037c060-6bd1-4aab-909e-d399d08481d5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:29.643975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d037c060-6bd1-4aab-909e-d399d08481d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:29.700767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1234/2511] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:29.709401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd1457-54d3-4ae9-ac93-dc8d4c74d4f4 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.709687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd1457-54d3-4ae9-ac93-dc8d4c74d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.710009 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd1457-54d3-4ae9-ac93-dc8d4c74d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.727996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-e1bd1457-54d3-4ae9-ac93-dc8d4c74d4f4 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:29.729205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e1bd1457-54d3-4ae9-ac93-dc8d4c74d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:29.737585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1278/2512] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:29.741544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037c060-6bd1-4aab-909e-d399d08481d5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:29.751780 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037c060-6bd1-4aab-909e-d399d08481d5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:29.756408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:29.757079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.757358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.757626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.758337 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:29.758635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:29.760209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.772080 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1235/2513] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:29.776105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd1457-54d3-4ae9-ac93-dc8d4c74d4f4 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:29.785497 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd1457-54d3-4ae9-ac93-dc8d4c74d4f4 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:29.790024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:29.790653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.790933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.791200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.791888 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:29.793784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.820510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1279/2514] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:29 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:29.824828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:29.835451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fad384-8c15-48bd-8368-0ff7bfb16051 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:29.893310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:29.893921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.894201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.894467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.895164 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:29.897210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.919084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1236/2515] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:29 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:29.921829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:29.932693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9f361-559c-4e93-a39c-e6de4c9b9c85 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:29.938144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:29.938801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.939100 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.939366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.940086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:29.941828 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.952331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1280/2516] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:29.956671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:29.967468 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886e032-95f1-4c3f-8373-5d5d25ba5972 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:29.972417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d48c-0083-4cbc-bcd5-83612183e4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:29.973078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d48c-0083-4cbc-bcd5-83612183e4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:29.973360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d48c-0083-4cbc-bcd5-83612183e4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:29.973632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d48c-0083-4cbc-bcd5-83612183e4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:29.974346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d48c-0083-4cbc-bcd5-83612183e4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:29.975923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d48c-0083-4cbc-bcd5-83612183e4c5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:29.990053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1237/2517] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:29.991230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:30.004449 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1088f81-090b-4241-b46a-131bb6ee4d48 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:30.009180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:30.010061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:30.010340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:30.010615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] PATH_INFO: `/v3/projects/c4fa50ed2e644c8c933657803b6a547d/users/d09f3ab53f494dd0a8a0d02396a6248b/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:30.025745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4fa50ed2e644c8c933657803b6a547d, user_id=d09f3ab53f494dd0a8a0d02396a6248b, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:30.028496 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:30.042430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1281/2518] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:29 2022] PUT /identity/v3/projects/c4fa50ed2e644c8c933657803b6a547d/users/d09f3ab53f494dd0a8a0d02396a6248b/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:30.056025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381806f-9429-4bd8-8b5a-274522d17e66 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:30.056313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381806f-9429-4bd8-8b5a-274522d17e66 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:30.056630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381806f-9429-4bd8-8b5a-274522d17e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:30.098000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8381806f-9429-4bd8-8b5a-274522d17e66 None None] MFA Rules not processed for user `d09f3ab53f494dd0a8a0d02396a6248b`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:30.099675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8381806f-9429-4bd8-8b5a-274522d17e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:30.125514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:30.181797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1238/2519] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:30.193679 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:30.195353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:30.260426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a768e-ad41-41d6-b408-b0eb1b0949d1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:30.264836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:30.265574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:30.265896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:30.266189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:30.275840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:30.277465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:30.289573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1282/2520] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:30.613378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:30.613672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:30.613948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:30.646032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] MFA Rules not processed for user `c26cd29ec472479a9a36ac9bb1f05580`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:30.656116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:30.722689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1239/2521] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:31.220338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:31.221786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:31.233286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:31.285187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17430909-e823-4732-bafd-c9af010756ec service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:31.288953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66c9be-a7c8-4d23-9039-a8ea8a509fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:31.289462 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66c9be-a7c8-4d23-9039-a8ea8a509fbc service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:31.289851 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66c9be-a7c8-4d23-9039-a8ea8a509fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:31.290043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66c9be-a7c8-4d23-9039-a8ea8a509fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:31.291801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1a66c9be-a7c8-4d23-9039-a8ea8a509fbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:31.302430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66c9be-a7c8-4d23-9039-a8ea8a509fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:31.303959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66c9be-a7c8-4d23-9039-a8ea8a509fbc service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:31.312572 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1283/2522] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:32.845292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:32.847222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:32.914881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4774e6-9561-4d4a-ad6c-f5ffb59fc522 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:32.921546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:32.922425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:32.922786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:32.923065 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] PATH_INFO: `/v3/users/3cb2514bd57747a59a19ac2e2d392670` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:32.924234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb2514bd57747a59a19ac2e2d392670)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:32.926183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:32.974327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb2514bd57747a59a19ac2e2d392670'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:32.974973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb2514bd57747a59a19ac2e2d392670'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:32.982024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb2514bd57747a59a19ac2e2d392670'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:32.982647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb2514bd57747a59a19ac2e2d392670'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:32.990003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb2514bd57747a59a19ac2e2d392670'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:32.995355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:33.009158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1240/2523] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:32 2022] DELETE /identity/v3/users/3cb2514bd57747a59a19ac2e2d392670 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:33.013784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:33.014279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:33.014870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:33.032643 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:33.033818 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:33.096086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1284/2524] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:33.105624 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:33.107272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:33.182102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8031d3-0635-4039-b3b6-928e3548ac7c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:33.185811 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:33.187150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:33.187491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:33.187776 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:33.196972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:33.198421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:33.208666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1241/2525] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:33.509591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:33.511031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:33.525693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead6431-c374-4d94-b97b-de039db8eca9 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:33.531506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:33.532403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:33.532692 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:33.532968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] PATH_INFO: `/v3/projects/27bd9552e272496d926c30ed3d5e591b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:33.534175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bd9552e272496d926c30ed3d5e591b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:33.536112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:33.583077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bd9552e272496d926c30ed3d5e591b', 'request_id': 'req-70262643-d067-4960-8477-bcf6c4896405'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.589140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bd9552e272496d926c30ed3d5e591b', 'request_id': 'req-70262643-d067-4960-8477-bcf6c4896405'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.595602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bd9552e272496d926c30ed3d5e591b', 'request_id': 'req-70262643-d067-4960-8477-bcf6c4896405'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.607989 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] The token cache is being invalidate because project 27bd9552e272496d926c30ed3d5e591b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:33.608904 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.609620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.611209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1285/2526] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:33 2022] DELETE /identity/v3/projects/27bd9552e272496d926c30ed3d5e591b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:33.615190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:33.625906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc691bf-5767-4989-a899-60007c38321b service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:33.681274 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:33.682137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:33.682417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:33.682684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] PATH_INFO: `/v3/users/45b165d521c74d77898607be65f685c2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:33.683841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b165d521c74d77898607be65f685c2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:33.685757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:33.730885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b165d521c74d77898607be65f685c2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.731520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b165d521c74d77898607be65f685c2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.739310 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b165d521c74d77898607be65f685c2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.739967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b165d521c74d77898607be65f685c2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.747060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-01ea465c-a74d-4310-968f-6565cb93e456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b165d521c74d77898607be65f685c2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:33.764584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1242/2527] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:33 2022] DELETE /identity/v3/users/45b165d521c74d77898607be65f685c2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:34.037501 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.038958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:34.052620 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70262643-d067-4960-8477-bcf6c4896405 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.086323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:34.113557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.114469 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.114702 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.114979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] PATH_INFO: `/v3/projects/c1b65ea8d65945e7afbcbef2381fd416` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.116243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b65ea8d65945e7afbcbef2381fd416)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.118206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.167356 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b65ea8d65945e7afbcbef2381fd416', 'request_id': 'req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:34.173551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b65ea8d65945e7afbcbef2381fd416', 'request_id': 'req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:34.179684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b65ea8d65945e7afbcbef2381fd416', 'request_id': 'req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:34.200182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] The token cache is being invalidate because project c1b65ea8d65945e7afbcbef2381fd416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:34.200543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:34.201158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-6cd6ee9b-b929-44c2-a36e-db0acdd3322a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:34.203445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1286/2528] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:34 2022] DELETE /identity/v3/projects/c1b65ea8d65945e7afbcbef2381fd416 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:34.208551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab9ad9-47c5-45ab-b2c6-e612802b8e74 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.209009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab9ad9-47c5-45ab-b2c6-e612802b8e74 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.209261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab9ad9-47c5-45ab-b2c6-e612802b8e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.227531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a2ab9ad9-47c5-45ab-b2c6-e612802b8e74 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:34.228854 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a2ab9ad9-47c5-45ab-b2c6-e612802b8e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:34.234518 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:34.300508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1243/2529] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:34.309674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739c0d-52fe-4d7c-865d-298cc33efa70 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.309973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739c0d-52fe-4d7c-865d-298cc33efa70 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.310239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739c0d-52fe-4d7c-865d-298cc33efa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.329853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-9e739c0d-52fe-4d7c-865d-298cc33efa70 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:34.331239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9e739c0d-52fe-4d7c-865d-298cc33efa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:34.339452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1287/2530] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:34.344255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab9ad9-47c5-45ab-b2c6-e612802b8e74 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.353941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab9ad9-47c5-45ab-b2c6-e612802b8e74 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.358775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.359427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.359709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.359985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.360734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:34.361040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.362671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.374207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1244/2531] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:34.378520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e739c0d-52fe-4d7c-865d-298cc33efa70 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.387774 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e739c0d-52fe-4d7c-865d-298cc33efa70 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.393004 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.393651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.393934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.394208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.394900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.396833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.424760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1288/2532] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:34 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:34.428879 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.439010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dd241-f349-48ec-a3d9-f9c145c9cd98 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.501263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.501978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.502262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.502539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.503233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.505251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.525383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1245/2533] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:34 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:34.529489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.540435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55eb89-26a3-4455-9834-82392243a101 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.545365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.546085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.546366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.546639 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.547359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.549025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.559310 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1289/2534] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:34.563346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.573387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e0a71-7b7a-4200-b725-d74b361f3468 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.578525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c00d3-82f3-40ae-a955-a65d4295ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.579179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c00d3-82f3-40ae-a955-a65d4295ed33 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.579462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c00d3-82f3-40ae-a955-a65d4295ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.579736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c00d3-82f3-40ae-a955-a65d4295ed33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.580507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c00d3-82f3-40ae-a955-a65d4295ed33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.582122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c00d3-82f3-40ae-a955-a65d4295ed33 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.591948 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1246/2535] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:34.596471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.606161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b459f27-806c-4366-9ae0-309bc9e902a9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.611055 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.611920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.612203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.612516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] PATH_INFO: `/v3/projects/12cd99c721a04d64a5546809a0606f2c/users/053d2fd644de469c879c9916074cf992/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.626767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12cd99c721a04d64a5546809a0606f2c, user_id=053d2fd644de469c879c9916074cf992, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.629381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.642240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1290/2536] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:34 2022] PUT /identity/v3/projects/12cd99c721a04d64a5546809a0606f2c/users/053d2fd644de469c879c9916074cf992/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:34.653177 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccaee3-7827-4d30-b55c-2b0c3099102b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.653469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccaee3-7827-4d30-b55c-2b0c3099102b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.653752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccaee3-7827-4d30-b55c-2b0c3099102b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.680479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-50ccaee3-7827-4d30-b55c-2b0c3099102b None None] MFA Rules not processed for user `053d2fd644de469c879c9916074cf992`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:34.681674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-50ccaee3-7827-4d30-b55c-2b0c3099102b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:34.747144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1247/2537] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:34.752069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.813232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13548950-1912-4ba5-8068-0bd2462b5b34 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.818856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.819523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.819853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.820387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.821695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.824053 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.831627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:34.855400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1291/2538] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:34 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:34.860272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccaee3-7827-4d30-b55c-2b0c3099102b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.866882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:34.871166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccaee3-7827-4d30-b55c-2b0c3099102b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.933770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.934437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.934717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.934989 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.935674 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.938530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.959256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1248/2539] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:34 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:34.963833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:34.976246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788704d-8c0e-4568-8644-16f60f0bb642 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:34.980920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:34.981830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:34.982094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:34.982364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:34.983099 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:34.985169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:34.995642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1292/2540] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:35.000321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:35.010298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5006c37-e9e0-44d8-94f9-495da293506b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:35.016195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:35.017102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:35.017384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:35.017660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] PATH_INFO: `/v3/projects/c1665463fafe402f9af5e72872ba82ea/users/d94d830aeb564724b39d3883d75e121b/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:35.032406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1665463fafe402f9af5e72872ba82ea, user_id=d94d830aeb564724b39d3883d75e121b, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:35.035017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:35.048653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1249/2541] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:34 2022] PUT /identity/v3/projects/c1665463fafe402f9af5e72872ba82ea/users/d94d830aeb564724b39d3883d75e121b/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:35.052525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:35.063318 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb070-5c00-4cd9-a68f-39488a372e14 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:35.137190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826af083-2500-4e0f-afda-573daa931e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:35.137864 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af083-2500-4e0f-afda-573daa931e9c admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:35.138589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af083-2500-4e0f-afda-573daa931e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:35.138926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af083-2500-4e0f-afda-573daa931e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:35.139762 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826af083-2500-4e0f-afda-573daa931e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:35.141677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826af083-2500-4e0f-afda-573daa931e9c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:35.152255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1293/2542] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:35.156391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:35.166549 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc16b8-63a8-41b7-9607-6e1c40771a68 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:35.171599 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:35.172508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:35.172787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:35.173057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] PATH_INFO: `/v3/projects/c1665463fafe402f9af5e72872ba82ea/users/d94d830aeb564724b39d3883d75e121b/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:35.174997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1665463fafe402f9af5e72872ba82ea, user_id=d94d830aeb564724b39d3883d75e121b, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:35.177631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:35.188371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1250/2543] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:35 2022] PUT /identity/v3/projects/c1665463fafe402f9af5e72872ba82ea/users/d94d830aeb564724b39d3883d75e121b/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:35.198173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:35.198461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:35.198736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:35.224725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] MFA Rules not processed for user `d94d830aeb564724b39d3883d75e121b`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:35.225913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:35.294889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1294/2544] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:35.307040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:35.311922 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:35.384830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c024c7-97f5-4791-9b66-41d2c0f031a0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:35.392930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:35.392930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:35.392930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:35.392930 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:35.399632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:35.401098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:35.411249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1251/2545] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:36.407697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:36.409206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:36.509951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79b8e9-563a-4945-98ca-13ffe4d03151 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:36.515136 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b9e76-cbdb-4852-a3e0-4463d88c24a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:36.515903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b9e76-cbdb-4852-a3e0-4463d88c24a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:36.516248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b9e76-cbdb-4852-a3e0-4463d88c24a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:36.516630 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b9e76-cbdb-4852-a3e0-4463d88c24a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:36.518589 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-949b9e76-cbdb-4852-a3e0-4463d88c24a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:36.534844 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:36.609273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b9e76-cbdb-4852-a3e0-4463d88c24a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:36.610750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b9e76-cbdb-4852-a3e0-4463d88c24a8 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:36.621843 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1295/2546] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:36.866457 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:36.932547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25f580-fe46-4f47-8792-ff9b126ba81e service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:36.937206 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:36.938082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:36.938360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:36.938631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] PATH_INFO: `/v3/users/053d2fd644de469c879c9916074cf992` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:36.939814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=053d2fd644de469c879c9916074cf992)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:36.941767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:36.943531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:36.996954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053d2fd644de469c879c9916074cf992'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:36.997638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053d2fd644de469c879c9916074cf992'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.005328 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053d2fd644de469c879c9916074cf992'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.005990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053d2fd644de469c879c9916074cf992'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.014927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053d2fd644de469c879c9916074cf992'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.033087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1252/2547] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:36 2022] DELETE /identity/v3/users/053d2fd644de469c879c9916074cf992 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:37.037664 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:37.037980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:37.038262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:37.057875 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:37.059110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:37.126746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1296/2548] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:37.137799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:37.139281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:37.199521 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe026c3-2d4d-4c69-87cd-0c08f043e352 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:37.204216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:37.204888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:37.205171 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:37.205447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:37.214704 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:37.216110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:37.226531 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1253/2549] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:37.688061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:37.689652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:37.769963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:37.772638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:37.805990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129bfee-eae0-4fb5-9e34-85fd551dbde2 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:37.815658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:37.816945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:37.817434 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:37.817985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] PATH_INFO: `/v3/projects/12cd99c721a04d64a5546809a0606f2c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:37.819695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=12cd99c721a04d64a5546809a0606f2c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:37.823052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:37.868109 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de8d9e-ecb0-458c-943d-d55058a63a80 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:37.875633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:37.876348 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:37.876975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:37.877422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:37.879590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:37.893510 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12cd99c721a04d64a5546809a0606f2c', 'request_id': 'req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.896935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:37.898522 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:37.901724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12cd99c721a04d64a5546809a0606f2c', 'request_id': 'req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.913847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12cd99c721a04d64a5546809a0606f2c', 'request_id': 'req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.922975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1297/2550] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:37.929843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] The token cache is being invalidate because project 12cd99c721a04d64a5546809a0606f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:37.930466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.931149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:37.936809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1254/2551] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:37 2022] DELETE /identity/v3/projects/12cd99c721a04d64a5546809a0606f2c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:37.938078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:37.957405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8050859-22ae-4067-98e6-c2cab53b2207 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:38.002681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:38.036231 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:38.037876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:38.037876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:38.037876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] PATH_INFO: `/v3/users/d94d830aeb564724b39d3883d75e121b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:38.039903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d94d830aeb564724b39d3883d75e121b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:38.041311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:38.102470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd94d830aeb564724b39d3883d75e121b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.103202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd94d830aeb564724b39d3883d75e121b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.104182 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd94d830aeb564724b39d3883d75e121b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.113148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd94d830aeb564724b39d3883d75e121b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.126163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1986d22b-15f8-4b58-8563-14b5679ed4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd94d830aeb564724b39d3883d75e121b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.151536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1298/2552] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:37 2022] DELETE /identity/v3/users/d94d830aeb564724b39d3883d75e121b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:38.442813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:38.444247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:38.454868 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:38.460125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a93ba-e7b0-4b6c-b461-44c44b35dcfb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:38.526860 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:38.527891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:38.528179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:38.528515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] PATH_INFO: `/v3/projects/c1665463fafe402f9af5e72872ba82ea` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:38.529902 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1665463fafe402f9af5e72872ba82ea)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:38.531826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:38.609081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1665463fafe402f9af5e72872ba82ea', 'request_id': 'req-133f2aef-e699-4509-a7a3-9b3d49be7052'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.616594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1665463fafe402f9af5e72872ba82ea', 'request_id': 'req-133f2aef-e699-4509-a7a3-9b3d49be7052'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.627645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1665463fafe402f9af5e72872ba82ea', 'request_id': 'req-133f2aef-e699-4509-a7a3-9b3d49be7052'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.667960 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] The token cache is being invalidate because project c1665463fafe402f9af5e72872ba82ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:38.668322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.669089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-133f2aef-e699-4509-a7a3-9b3d49be7052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:38.670936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1255/2553] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:38 2022] DELETE /identity/v3/projects/c1665463fafe402f9af5e72872ba82ea => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:38.675842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edec2b-eed3-4217-bc16-98a706ee835c None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:38.676218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edec2b-eed3-4217-bc16-98a706ee835c None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:38.676584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edec2b-eed3-4217-bc16-98a706ee835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:38.695072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a6edec2b-eed3-4217-bc16-98a706ee835c None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:38.696324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a6edec2b-eed3-4217-bc16-98a706ee835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:38.783830 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1299/2554] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:38.793458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50abb-e9a2-4873-b210-056bc66d4cf7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:38.793754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50abb-e9a2-4873-b210-056bc66d4cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:38.794045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce50abb-e9a2-4873-b210-056bc66d4cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:38.814483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2ce50abb-e9a2-4873-b210-056bc66d4cf7 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:38.815803 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2ce50abb-e9a2-4873-b210-056bc66d4cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:38.823937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1256/2555] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:38.830050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edec2b-eed3-4217-bc16-98a706ee835c None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:38.840466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edec2b-eed3-4217-bc16-98a706ee835c None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:38.846110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:38.846958 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:38.847234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:38.847503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:38.848230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:38.848566 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:38.850389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:38.863082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1300/2556] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:38.867533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50abb-e9a2-4873-b210-056bc66d4cf7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:38.878067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce50abb-e9a2-4873-b210-056bc66d4cf7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:38.883898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:38.884626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:38.884909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:38.885188 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:38.886010 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:38.887941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:38.919517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1257/2557] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:38 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:38.921158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:38.931353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9829a38-a0c9-4f83-83a3-311fd7717146 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:38.976495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:38.992090 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:38.992788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:38.993068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:38.993338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:38.994028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:38.995916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.016218 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1301/2558] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:38 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:39.020471 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.031503 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888574ac-79ac-421a-8f46-ab0bf442ced7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.036075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.036762 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.037044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.037322 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.038049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.039623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.049977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1258/2559] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:39.053767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.063724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd476b-501f-463e-bbf5-58ff5dbce18e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.068304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe83973-a8b0-4edf-826a-ba7d000990f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.068929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83973-a8b0-4edf-826a-ba7d000990f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.069208 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83973-a8b0-4edf-826a-ba7d000990f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.069482 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83973-a8b0-4edf-826a-ba7d000990f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.070232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe83973-a8b0-4edf-826a-ba7d000990f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.071797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe83973-a8b0-4edf-826a-ba7d000990f5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.083485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1302/2560] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:39.087174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.098110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f74bc6-f0e6-4459-b0e1-ff3d3bb8926e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.102699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.103576 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.103857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.104132 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] PATH_INFO: `/v3/projects/dafa818b25ad45c8b734acf34dd05f5f/users/1602be695b3142d48bbadf9bc1feea6c/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.120855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafa818b25ad45c8b734acf34dd05f5f, user_id=1602be695b3142d48bbadf9bc1feea6c, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.124888 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.142136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1259/2561] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:39 2022] PUT /identity/v3/projects/dafa818b25ad45c8b734acf34dd05f5f/users/1602be695b3142d48bbadf9bc1feea6c/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:39.152214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de759dfd-acf3-4c4e-a950-10f8b51119c7 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.152488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de759dfd-acf3-4c4e-a950-10f8b51119c7 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.152763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de759dfd-acf3-4c4e-a950-10f8b51119c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.180176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-de759dfd-acf3-4c4e-a950-10f8b51119c7 None None] MFA Rules not processed for user `1602be695b3142d48bbadf9bc1feea6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:39.181457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-de759dfd-acf3-4c4e-a950-10f8b51119c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:39.248262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1303/2562] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:39.253333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.288207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:39.318772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745332f-5806-4083-bf6c-3a8f6d045624 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.323312 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.323967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.324250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.324554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.325247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.327725 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.351729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1260/2563] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:39 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:39.355815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de759dfd-acf3-4c4e-a950-10f8b51119c7 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.366834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de759dfd-acf3-4c4e-a950-10f8b51119c7 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.424476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.425152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.425433 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.425710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.426407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.428312 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.447354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1304/2564] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:39 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:39.451375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.461677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2418d-b089-484d-825b-3d9b18d5e282 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.466397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.467067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.467837 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.468110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.468848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.470444 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.480814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1261/2565] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:39.486027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.495738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e174892-a6b7-42c1-928e-5e64e074efee admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.500506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.501369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.501647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.502003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] PATH_INFO: `/v3/projects/699d9123e21340f1bc23c0b875b42b7d/users/6a9251dc0a714767a75985ec24c3af58/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.517161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=699d9123e21340f1bc23c0b875b42b7d, user_id=6a9251dc0a714767a75985ec24c3af58, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.519822 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.533010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1305/2566] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:39 2022] PUT /identity/v3/projects/699d9123e21340f1bc23c0b875b42b7d/users/6a9251dc0a714767a75985ec24c3af58/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:39.536889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.547299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d13c7-885b-4a5c-98ac-0c9afa3fb05b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.602197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05565b9-5251-4329-8778-86c32bfbfd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.602783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05565b9-5251-4329-8778-86c32bfbfd9f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.603064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05565b9-5251-4329-8778-86c32bfbfd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.603339 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05565b9-5251-4329-8778-86c32bfbfd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.604054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05565b9-5251-4329-8778-86c32bfbfd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.605677 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05565b9-5251-4329-8778-86c32bfbfd9f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.615571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1262/2567] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:39.619866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.630181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7cc1c-2f1c-495d-8e32-27b3ef83c9eb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.634789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.635667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.635946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.636220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] PATH_INFO: `/v3/projects/699d9123e21340f1bc23c0b875b42b7d/users/6a9251dc0a714767a75985ec24c3af58/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.638235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699d9123e21340f1bc23c0b875b42b7d, user_id=6a9251dc0a714767a75985ec24c3af58, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.641555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.651418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1306/2568] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:39 2022] PUT /identity/v3/projects/699d9123e21340f1bc23c0b875b42b7d/users/6a9251dc0a714767a75985ec24c3af58/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:39.662258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a25a-119e-44a1-8a00-002195f68480 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.662566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a25a-119e-44a1-8a00-002195f68480 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.662847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a25a-119e-44a1-8a00-002195f68480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.689379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3950a25a-119e-44a1-8a00-002195f68480 None None] MFA Rules not processed for user `6a9251dc0a714767a75985ec24c3af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:39.690569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3950a25a-119e-44a1-8a00-002195f68480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:39.757855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1263/2569] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:39.770386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:39.772476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:39.779248 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:39.846722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80ec4c-b7bc-4f8f-89e9-180c0ff54ae7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:39.850366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:39.850999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:39.851276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:39.851546 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:39.861647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:39.863068 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:39.872614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1307/2570] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:40.183011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a25a-119e-44a1-8a00-002195f68480 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:40.192085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a25a-119e-44a1-8a00-002195f68480 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:40.196664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a9251dc0a714767a75985ec24c3af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '699d9123e21340f1bc23c0b875b42b7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:40.197313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:40.197597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:40.197875 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:40.198597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:40.198904 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:40.200534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:40.212079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1264/2571] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:40.216033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:40.226106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486b8dd-2783-4f02-9ea9-9a3501f79dd8 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:40.230727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a9251dc0a714767a75985ec24c3af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '699d9123e21340f1bc23c0b875b42b7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:40.231367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:40.231903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:40.232178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:40.232927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:40.235540 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:40.261073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1308/2572] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:40 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:40.265235 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:40.275384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b73e51-1dd9-48c6-bf32-8614543eea06 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:40.285092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:40.336289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a9251dc0a714767a75985ec24c3af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '699d9123e21340f1bc23c0b875b42b7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:40.337034 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:40.337327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:40.337613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:40.338372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:40.338671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:40.340275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:40.352589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1265/2573] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 16:05:40.391918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:40.402379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aff405-3723-4dc1-bbfb-7635c62c6f77 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:40.407651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a9251dc0a714767a75985ec24c3af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '699d9123e21340f1bc23c0b875b42b7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:40.408612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:40.408891 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:40.409162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] PATH_INFO: `/v3/projects/531424e5ef12497fa924359e33ab8377` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:40.410369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorizing `identity:delete_project(project_id=531424e5ef12497fa924359e33ab8377)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:40.412222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:40.471486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531424e5ef12497fa924359e33ab8377', 'request_id': 'req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.478150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531424e5ef12497fa924359e33ab8377', 'request_id': 'req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.485332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531424e5ef12497fa924359e33ab8377', 'request_id': 'req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.500790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] The token cache is being invalidate because project 531424e5ef12497fa924359e33ab8377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:40.501066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.501728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5a566adf-69f7-4fee-aa5c-d7a35af29fb6 tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.503296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1309/2574] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:40 2022] DELETE /identity/v3/projects/531424e5ef12497fa924359e33ab8377 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:40.690307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:40.692011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:40.758167 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225013-aeae-4edb-9b57-64d8c28bfcab tempest-VolumeQuotaClassesTest-945735645 tempest-VolumeQuotaClassesTest-945735645-project-admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:40.820146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:40.821045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:40.821401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:40.821676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] PATH_INFO: `/v3/users/1602be695b3142d48bbadf9bc1feea6c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:40.822856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1602be695b3142d48bbadf9bc1feea6c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:40.824783 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:40.872564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1602be695b3142d48bbadf9bc1feea6c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.873224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1602be695b3142d48bbadf9bc1feea6c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.880917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1602be695b3142d48bbadf9bc1feea6c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.881691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1602be695b3142d48bbadf9bc1feea6c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.883407 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:40.891238 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1602be695b3142d48bbadf9bc1feea6c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:40.911689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1266/2575] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:40 2022] DELETE /identity/v3/users/1602be695b3142d48bbadf9bc1feea6c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:40.916028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:40.916531 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:40.916671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:40.934266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:40.935421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:40.999569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1310/2576] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:41.009629 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:41.011038 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:41.070652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cd52f-b266-4795-93ee-e4adc9a38709 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:41.074336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:41.074982 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:41.075267 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:41.075545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:41.085159 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:41.086589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:41.098174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1267/2577] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:41.600275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:41.601826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:41.614371 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c743ca-0e01-4efb-87a3-341d9187327d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:41.619716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:41.620590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:41.620966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:41.621181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] PATH_INFO: `/v3/projects/dafa818b25ad45c8b734acf34dd05f5f` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:41.622388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=dafa818b25ad45c8b734acf34dd05f5f)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:41.631457 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:41.640543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:41.758080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dafa818b25ad45c8b734acf34dd05f5f', 'request_id': 'req-b3f7e6eb-2967-4440-a994-53af1913b5df'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.765162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dafa818b25ad45c8b734acf34dd05f5f', 'request_id': 'req-b3f7e6eb-2967-4440-a994-53af1913b5df'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.772848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dafa818b25ad45c8b734acf34dd05f5f', 'request_id': 'req-b3f7e6eb-2967-4440-a994-53af1913b5df'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.795166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] The token cache is being invalidate because project dafa818b25ad45c8b734acf34dd05f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:41.796295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.797159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.799198 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1311/2578] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:41 2022] DELETE /identity/v3/projects/dafa818b25ad45c8b734acf34dd05f5f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:41.803066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:41.814663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59072004-51e9-4d24-88a4-d7361b821af8 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:41.883972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:41.887750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:41.888051 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:41.888417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] PATH_INFO: `/v3/users/6a9251dc0a714767a75985ec24c3af58` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:41.890078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9251dc0a714767a75985ec24c3af58)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:41.892167 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:41.951430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9251dc0a714767a75985ec24c3af58'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.952379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9251dc0a714767a75985ec24c3af58'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.960036 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9251dc0a714767a75985ec24c3af58'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.960658 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9251dc0a714767a75985ec24c3af58'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.968734 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-38e49f53-96f2-4ae2-9b63-c763940945d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9251dc0a714767a75985ec24c3af58'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:41.988790 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:42.004166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1268/2579] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:41 2022] DELETE /identity/v3/users/6a9251dc0a714767a75985ec24c3af58 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:43.095868 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:43.101645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:43.125447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7e6eb-2967-4440-a994-53af1913b5df admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:43.237158 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:43.239050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.239611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.240327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] PATH_INFO: `/v3/projects/699d9123e21340f1bc23c0b875b42b7d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.242535 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699d9123e21340f1bc23c0b875b42b7d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:43.245034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:43.399899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699d9123e21340f1bc23c0b875b42b7d', 'request_id': 'req-770bb417-b569-48d9-aa7a-dbcd225c6394'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:43.433833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699d9123e21340f1bc23c0b875b42b7d', 'request_id': 'req-770bb417-b569-48d9-aa7a-dbcd225c6394'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:43.443108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699d9123e21340f1bc23c0b875b42b7d', 'request_id': 'req-770bb417-b569-48d9-aa7a-dbcd225c6394'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:43.461731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] The token cache is being invalidate because project 699d9123e21340f1bc23c0b875b42b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:43.462246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:43.463878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-770bb417-b569-48d9-aa7a-dbcd225c6394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:43.468447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1312/2580] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:43 2022] DELETE /identity/v3/projects/699d9123e21340f1bc23c0b875b42b7d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:43.474192 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f7e4-ac06-4ce4-b80e-3718a62dd873 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.474634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f7e4-ac06-4ce4-b80e-3718a62dd873 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.474728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f7e4-ac06-4ce4-b80e-3718a62dd873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.508520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-97c5f7e4-ac06-4ce4-b80e-3718a62dd873 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:43.511021 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-97c5f7e4-ac06-4ce4-b80e-3718a62dd873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:43.599336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1269/2581] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:43.608548 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e0143-1172-42a7-99c6-f82ea03ca917 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.608832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e0143-1172-42a7-99c6-f82ea03ca917 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.609657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e0143-1172-42a7-99c6-f82ea03ca917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.645201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-b39e0143-1172-42a7-99c6-f82ea03ca917 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:43.646833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b39e0143-1172-42a7-99c6-f82ea03ca917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:43.660233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1313/2582] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:43.664405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5f7e4-ac06-4ce4-b80e-3718a62dd873 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:43.676133 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5f7e4-ac06-4ce4-b80e-3718a62dd873 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:43.682795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:43.683553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.683965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.684272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.685190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:43.685528 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:43.688053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:43.707980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1270/2583] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:43.712120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e0143-1172-42a7-99c6-f82ea03ca917 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:43.722534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e0143-1172-42a7-99c6-f82ea03ca917 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:43.729082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:43.729900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.730187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.730516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.731439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:43.734602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:43.750942 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:43.772132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1314/2584] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:43 2022] POST /identity/v3/projects => generated 361 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:43.776089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:43.787877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb111b85-90ff-41c8-845c-f52d22034a1b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:43.848589 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:43.849583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.849895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.850167 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.850884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:43.852842 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:43.876053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1271/2585] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:43 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:43.880669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:43.892019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460f5e-b09e-446d-ae5a-ac224a7aa208 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:43.896806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:43.897464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.897860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.898063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.899137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:43.900659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:43.914979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1315/2586] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:43.916411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:43.929285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86365d07-f6cd-455a-b920-9c621948f929 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:43.934863 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45779915-4df2-4fab-8db6-7ea095d372af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:43.935632 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45779915-4df2-4fab-8db6-7ea095d372af admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.936500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45779915-4df2-4fab-8db6-7ea095d372af admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.936858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45779915-4df2-4fab-8db6-7ea095d372af admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:43.937659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45779915-4df2-4fab-8db6-7ea095d372af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:43.939346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45779915-4df2-4fab-8db6-7ea095d372af admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:43.957123 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1272/2587] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:43.960219 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:43.975527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9baf04-2fea-4200-97d4-6ac03be72173 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:43.980407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:43.981551 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:43.982096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:43.982351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] PATH_INFO: `/v3/projects/f96a974b51a8403d89d4bebb2f5e3bcb/users/521479b42278443bb1537b7356228a91/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:44.001144 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96a974b51a8403d89d4bebb2f5e3bcb, user_id=521479b42278443bb1537b7356228a91, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:44.003803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:44.021705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1316/2588] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:43 2022] PUT /identity/v3/projects/f96a974b51a8403d89d4bebb2f5e3bcb/users/521479b42278443bb1537b7356228a91/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:44.033622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c0a7-d894-4ab5-befe-c87c897d2da3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:44.034117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c0a7-d894-4ab5-befe-c87c897d2da3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:44.034223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c0a7-d894-4ab5-befe-c87c897d2da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:44.093358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6fc3c0a7-d894-4ab5-befe-c87c897d2da3 None None] MFA Rules not processed for user `521479b42278443bb1537b7356228a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:44.094946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6fc3c0a7-d894-4ab5-befe-c87c897d2da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:44.199559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:44.220596 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1273/2589] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:44.567876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:44.569658 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:44.697869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e4cf-0d56-4e7c-9497-68b7a35f05b7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:44.703339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df673a3-2cf3-43fb-9e9f-d3a4d59f4d00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:44.704529 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df673a3-2cf3-43fb-9e9f-d3a4d59f4d00 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:44.705191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df673a3-2cf3-43fb-9e9f-d3a4d59f4d00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:44.706081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df673a3-2cf3-43fb-9e9f-d3a4d59f4d00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:44.717609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df673a3-2cf3-43fb-9e9f-d3a4d59f4d00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:44.719453 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df673a3-2cf3-43fb-9e9f-d3a4d59f4d00 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:44.737203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1317/2590] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:44.755419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c0a7-d894-4ab5-befe-c87c897d2da3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:44.965895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c0a7-d894-4ab5-befe-c87c897d2da3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:44.974296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:44.975567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:44.976053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:44.976849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] PATH_INFO: `/v3/users/521479b42278443bb1537b7356228a91` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:44.979055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521479b42278443bb1537b7356228a91)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:44.982294 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:45.085213 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521479b42278443bb1537b7356228a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.085970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521479b42278443bb1537b7356228a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.095425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521479b42278443bb1537b7356228a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.096538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521479b42278443bb1537b7356228a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.105864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521479b42278443bb1537b7356228a91'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.169726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1274/2591] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:44 2022] DELETE /identity/v3/users/521479b42278443bb1537b7356228a91 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:45.175565 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:45.175902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:45.176174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:45.204786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:45.205866 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:45.301748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1318/2592] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:45.315865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:45.317460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:45.391110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576a4f3-4ff4-4a3b-be4a-242e8b936ca5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:45.394787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:45.395446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:45.395724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:45.396012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:45.406813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:45.407839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:45.419554 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1275/2593] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:45.781869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:45.784780 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:45.801353 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552d46b-fad1-49ac-bb1e-1f7e3090b5bb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:45.806840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:45.807923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:45.808465 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:45.819192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] PATH_INFO: `/v3/projects/f96a974b51a8403d89d4bebb2f5e3bcb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:45.821394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96a974b51a8403d89d4bebb2f5e3bcb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:45.823998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:45.923257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:45.932250 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b6f60-5e0c-437b-9156-6680fb138370 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:45.934637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96a974b51a8403d89d4bebb2f5e3bcb', 'request_id': 'req-75d517b1-a954-4e2c-b040-ade23e33a574'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.940582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:45.942281 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96a974b51a8403d89d4bebb2f5e3bcb', 'request_id': 'req-75d517b1-a954-4e2c-b040-ade23e33a574'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.944276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:45.950961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96a974b51a8403d89d4bebb2f5e3bcb', 'request_id': 'req-75d517b1-a954-4e2c-b040-ade23e33a574'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.968951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] The token cache is being invalidate because project f96a974b51a8403d89d4bebb2f5e3bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:45.969237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.970386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-75d517b1-a954-4e2c-b040-ade23e33a574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:45.972199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1319/2594] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:45 2022] DELETE /identity/v3/projects/f96a974b51a8403d89d4bebb2f5e3bcb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:45.977154 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873dfb3-feb2-46b7-b54f-330038d472bc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:45.977677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873dfb3-feb2-46b7-b54f-330038d472bc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:45.978033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873dfb3-feb2-46b7-b54f-330038d472bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:45.998201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-7873dfb3-feb2-46b7-b54f-330038d472bc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:46.000013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-7873dfb3-feb2-46b7-b54f-330038d472bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.006091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.007242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.007493 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.007786 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.010479 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.078602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1320/2595] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:46.087009 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baae444-63a5-4321-8143-bc29148e1cf6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.087304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baae444-63a5-4321-8143-bc29148e1cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.087582 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baae444-63a5-4321-8143-bc29148e1cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.122398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3baae444-63a5-4321-8143-bc29148e1cf6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:46.124634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3baae444-63a5-4321-8143-bc29148e1cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.136984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1321/2596] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:46.139845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baae444-63a5-4321-8143-bc29148e1cf6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.143538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.145193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.157346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1276/2597] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:46.161667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baae444-63a5-4321-8143-bc29148e1cf6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.169480 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.170980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.171552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.171834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.172606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:46.172919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.174536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.188404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1322/2598] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:46.191773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.204001 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337c73a-9ae5-4216-ae10-99abba3c30be service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.209292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.209965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.210248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.210525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.211220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.213201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.261754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1277/2599] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:46 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:46.265661 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.279290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710fe6-b200-489d-8772-14f35e4d010f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.379225 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.380842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.381576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.381576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.382421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.386236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.437929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1323/2600] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:46 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:46.439152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.454202 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13866-4802-455a-ba00-5574f0073d31 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.471743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.472642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.472861 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.476236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.476236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.476236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.477290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.482151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.493921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1278/2601] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:46.495945 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.501700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a2c3d-017d-46ce-bcca-641e6c792de7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.506732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.507610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.508399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.508536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] PATH_INFO: `/v3/users/dcd51c4ab0a54afc8a60c83aac101fe3` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.509747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd51c4ab0a54afc8a60c83aac101fe3)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.513403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.522591 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b686-8748-470c-a978-3e0bb18811b0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.529405 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.530600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.530885 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.531160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.531884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.533679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.546071 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1279/2602] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:46.552797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.570430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77b8d0-415c-4eca-a5d7-69c6983fc50a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.571475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd51c4ab0a54afc8a60c83aac101fe3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.572199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd51c4ab0a54afc8a60c83aac101fe3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.572980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd51c4ab0a54afc8a60c83aac101fe3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.575949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:46.578871 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a4e69-bc52-434f-a490-faef15725ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.579771 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4e69-bc52-434f-a490-faef15725ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.580128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4e69-bc52-434f-a490-faef15725ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.580696 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4e69-bc52-434f-a490-faef15725ec7 admin admin] PATH_INFO: `/v3/projects/99bd6743619b49879367d0c512fca455/users/648495fd539d4cdc9493bef9576415a5/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.583294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd51c4ab0a54afc8a60c83aac101fe3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.599045 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a4e69-bc52-434f-a490-faef15725ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bd6743619b49879367d0c512fca455, user_id=648495fd539d4cdc9493bef9576415a5, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.601864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a4e69-bc52-434f-a490-faef15725ec7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.602687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-3b4f09dd-9345-4e18-b90b-02970fa364f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd51c4ab0a54afc8a60c83aac101fe3'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.618113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1280/2603] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:46 2022] PUT /identity/v3/projects/99bd6743619b49879367d0c512fca455/users/648495fd539d4cdc9493bef9576415a5/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:46.618626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1324/2604] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:46 2022] DELETE /identity/v3/users/dcd51c4ab0a54afc8a60c83aac101fe3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:46.622946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.624099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.624608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.629108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.629505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.629912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.644621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:46.645834 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.663283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] MFA Rules not processed for user `648495fd539d4cdc9493bef9576415a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:46.664634 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.700186 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:46.715460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1281/2605] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:46.725824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.727991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.736948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1325/2606] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:46.747980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.749925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.802023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5abcb-6709-4d79-b345-b366e89c09bf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.805920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.806743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.807099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.807445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.807569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff9ea4-1630-41ee-9d88-776f60f829e1 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.812252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a530354-34fb-49a7-b71b-e7ce3853c4dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.812252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a530354-34fb-49a7-b71b-e7ce3853c4dc service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.812252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a530354-34fb-49a7-b71b-e7ce3853c4dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.812557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a530354-34fb-49a7-b71b-e7ce3853c4dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.818593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.820178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.821955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a530354-34fb-49a7-b71b-e7ce3853c4dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.823367 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a530354-34fb-49a7-b71b-e7ce3853c4dc service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.831974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1282/2607] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:46.832595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1326/2608] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:46.857141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:46.859252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:46.872611 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc1449-54a6-45bd-9a5a-d134b3768454 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:46.878367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:46.879276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.879560 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.879828 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] PATH_INFO: `/v3/users/648495fd539d4cdc9493bef9576415a5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:46.881100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648495fd539d4cdc9493bef9576415a5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:46.883047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:46.942713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648495fd539d4cdc9493bef9576415a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.943862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648495fd539d4cdc9493bef9576415a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.953843 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648495fd539d4cdc9493bef9576415a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.954543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648495fd539d4cdc9493bef9576415a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.965958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648495fd539d4cdc9493bef9576415a5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:46.990580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1283/2609] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:46 2022] DELETE /identity/v3/users/648495fd539d4cdc9493bef9576415a5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:46.995244 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:46.995610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:46.995974 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:47.017149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:47.018383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:47.093430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:47.116855 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1327/2610] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:47.132396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:47.132866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:47.186344 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:47.187948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:47.201273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbc29c-8a98-44ec-9ffb-25a0f9902ff2 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:47.206797 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:47.207705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:47.208062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:47.208452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] PATH_INFO: `/v3/projects/aaad9a77b56043de8b38450ec47ecb0e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:47.209819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaad9a77b56043de8b38450ec47ecb0e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:47.211808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:47.214749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26825b8f-ffa1-47ee-aa09-4338321f9193 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:47.219721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:47.220024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:47.220307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:47.220631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:47.232031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:47.233941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:47.245282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1284/2611] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:47.276831 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaad9a77b56043de8b38450ec47ecb0e', 'request_id': 'req-13875ab9-0c58-465a-9877-137473770867'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.285124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaad9a77b56043de8b38450ec47ecb0e', 'request_id': 'req-13875ab9-0c58-465a-9877-137473770867'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.292514 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaad9a77b56043de8b38450ec47ecb0e', 'request_id': 'req-13875ab9-0c58-465a-9877-137473770867'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.311060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] The token cache is being invalidate because project aaad9a77b56043de8b38450ec47ecb0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:47.311505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.313646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.317395 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1328/2612] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:47 2022] DELETE /identity/v3/projects/aaad9a77b56043de8b38450ec47ecb0e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:47.321709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:47.333964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c66d2-5d8f-4e58-9314-6baf616b1b1a service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:47.346607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:47.406193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:47.407087 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:47.407371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:47.407647 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] PATH_INFO: `/v3/users/f1b84fb86b674c7ab0bb4091a4685ab5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:47.409128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b84fb86b674c7ab0bb4091a4685ab5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:47.411395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:47.461443 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b84fb86b674c7ab0bb4091a4685ab5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.462308 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b84fb86b674c7ab0bb4091a4685ab5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.473104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b84fb86b674c7ab0bb4091a4685ab5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.473813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b84fb86b674c7ab0bb4091a4685ab5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.482286 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b84fb86b674c7ab0bb4091a4685ab5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.505416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1285/2613] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:47 2022] DELETE /identity/v3/users/f1b84fb86b674c7ab0bb4091a4685ab5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:47.791684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:47.793506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:47.809369 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13875ab9-0c58-465a-9877-137473770867 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:47.825388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:47.828020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:47.882050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:47.882877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:47.883155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:47.883428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] PATH_INFO: `/v3/projects/99bd6743619b49879367d0c512fca455` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:47.884981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bd6743619b49879367d0c512fca455)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:47.886917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:47.943014 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606eee5-e72c-4e21-8a1b-bca81773f45f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:47.949417 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bd6743619b49879367d0c512fca455', 'request_id': 'req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.963700 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bd6743619b49879367d0c512fca455', 'request_id': 'req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.972924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bd6743619b49879367d0c512fca455', 'request_id': 'req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.992550 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] The token cache is being invalidate because project 99bd6743619b49879367d0c512fca455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:47.992920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.993613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-09fa2836-a692-4fb6-a4a4-d3546ef21f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:47.995474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1329/2614] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:47 2022] DELETE /identity/v3/projects/99bd6743619b49879367d0c512fca455 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:48.001355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c164a-e5ad-43ef-821d-0f32a0c50e10 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.001355 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c164a-e5ad-43ef-821d-0f32a0c50e10 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.001597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c164a-e5ad-43ef-821d-0f32a0c50e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.006662 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:48.028125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.029039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.029319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.030306 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] PATH_INFO: `/v3/projects/dd0b759e78d54419800ec141a9dfdc68` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.031988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0b759e78d54419800ec141a9dfdc68)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.035661 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.039957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-1b0c164a-e5ad-43ef-821d-0f32a0c50e10 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:48.041352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1b0c164a-e5ad-43ef-821d-0f32a0c50e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:48.057069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1330/2615] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:48.059026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1df9f-856d-4cec-98b8-2a91d3cd1ebb None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.059384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1df9f-856d-4cec-98b8-2a91d3cd1ebb None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.059717 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1df9f-856d-4cec-98b8-2a91d3cd1ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.075571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:48.082752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-43b1df9f-856d-4cec-98b8-2a91d3cd1ebb None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:48.084960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-43b1df9f-856d-4cec-98b8-2a91d3cd1ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:48.102315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0b759e78d54419800ec141a9dfdc68', 'request_id': 'req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:48.120598 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0b759e78d54419800ec141a9dfdc68', 'request_id': 'req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:48.141166 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0b759e78d54419800ec141a9dfdc68', 'request_id': 'req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:48.157195 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] The token cache is being invalidate because project dd0b759e78d54419800ec141a9dfdc68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:48.157559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:48.158272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-27a6ba0e-cb07-4da9-bf2f-4d1aa97f5e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:48.160096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1286/2616] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:47 2022] DELETE /identity/v3/projects/dd0b759e78d54419800ec141a9dfdc68 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:48.164631 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098b0d3-a40d-48c5-a3aa-25acf1a9aec3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.164986 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098b0d3-a40d-48c5-a3aa-25acf1a9aec3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.165333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098b0d3-a40d-48c5-a3aa-25acf1a9aec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.180045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1331/2617] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:48.184326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1df9f-856d-4cec-98b8-2a91d3cd1ebb None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.189613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e098b0d3-a40d-48c5-a3aa-25acf1a9aec3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:48.191545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e098b0d3-a40d-48c5-a3aa-25acf1a9aec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:48.194156 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1df9f-856d-4cec-98b8-2a91d3cd1ebb None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.198802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.199451 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.199732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.200008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.200237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1287/2618] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:48.200751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:48.201052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.202957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.207749 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab8448-ddcb-4475-bc89-abb3541a3dbf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.208032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab8448-ddcb-4475-bc89-abb3541a3dbf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.208307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab8448-ddcb-4475-bc89-abb3541a3dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.215012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1332/2619] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 16:05:48.219811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.228063 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f6ab8448-ddcb-4475-bc89-abb3541a3dbf None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:48.229207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f6ab8448-ddcb-4475-bc89-abb3541a3dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:48.229575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b3cb7-0c23-4dc1-80d5-454a5bd8ef8a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.235468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.236383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.236708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.236949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.237624 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.239519 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.240498 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1288/2620] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:48.243282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab8448-ddcb-4475-bc89-abb3541a3dbf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.254926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab8448-ddcb-4475-bc89-abb3541a3dbf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.260535 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.261194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.261483 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.261759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.262814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:48.263030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.265824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.272400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1333/2621] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:48.276662 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.284637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1289/2622] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 16:05:48.286501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.297000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cae10-a3e7-4e36-bb70-b77453d17ff8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.352642 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5429b-b252-4741-af50-af97156f0f92 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.358130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.359000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.359000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.359271 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.360009 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.361943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.367149 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.367906 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.368196 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.368609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.369375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.371396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.385934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1334/2623] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:48.390545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.401436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495626d1-c9cd-480e-ae35-f25788c6374a admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.407273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.410848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1290/2624] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:48.410941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.410941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.410941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.410941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.410941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.411601 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.424074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1335/2625] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:48.427836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.441021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1f9ac-64d7-42bf-b291-86b6dd31335e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.445652 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:48.487731 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56799b6-6db3-47d9-900d-d6b66043247f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.492789 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.493486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.493782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.494056 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.494937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.496962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.499575 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:48.517464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.518301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.518766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.519167 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.520007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.526322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.532534 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1291/2626] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:48.534214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.544995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1336/2627] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:48.546055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7f68e-e217-4419-89d3-2355edf71057 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.550995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.554977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.556437 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.557285 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.557970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.559501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.562673 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.566162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f73da-70ac-4742-a2d9-b93b31ccbb36 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.572916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.574335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.574771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.575266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] PATH_INFO: `/v3/projects/7a5590f870754829924beac551ba3742/users/c16cee31cb334a84aa14482bbb494664/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.581779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1292/2628] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:48.583564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.601519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133b25e-d637-449e-baaa-0c4ec3b86db0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.605733 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5590f870754829924beac551ba3742, user_id=c16cee31cb334a84aa14482bbb494664, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.608211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a2162-cdae-43a7-82f7-dc776a85c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.609794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.610132 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2162-cdae-43a7-82f7-dc776a85c8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.610574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2162-cdae-43a7-82f7-dc776a85c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.611008 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2162-cdae-43a7-82f7-dc776a85c8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.611882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a2162-cdae-43a7-82f7-dc776a85c8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.615289 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a2162-cdae-43a7-82f7-dc776a85c8d9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.640936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1337/2629] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:48 2022] PUT /identity/v3/projects/7a5590f870754829924beac551ba3742/users/c16cee31cb334a84aa14482bbb494664/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:48.641148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1293/2630] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:48.641351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.643261 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.643561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.643844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.694664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] MFA Rules not processed for user `c16cee31cb334a84aa14482bbb494664`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:48.696024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:48.717330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6f091-59be-48a6-8ef8-9ce162dadd05 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.723577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dba1e-e065-41ae-a230-fd7e155d49db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.724505 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dba1e-e065-41ae-a230-fd7e155d49db admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.724789 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dba1e-e065-41ae-a230-fd7e155d49db admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.725063 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dba1e-e065-41ae-a230-fd7e155d49db admin admin] PATH_INFO: `/v3/projects/a9a04292be0743af8c9acbeb20441416/users/c6358240b0084c0bbc475e7db74f74a0/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.751613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dba1e-e065-41ae-a230-fd7e155d49db admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a04292be0743af8c9acbeb20441416, user_id=c6358240b0084c0bbc475e7db74f74a0, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:48.754249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dba1e-e065-41ae-a230-fd7e155d49db admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:48.784447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1338/2631] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:48 2022] PUT /identity/v3/projects/a9a04292be0743af8c9acbeb20441416/users/c6358240b0084c0bbc475e7db74f74a0/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:48.786327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094d5e-ac66-4962-a96e-911c619d1cc6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.786612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094d5e-ac66-4962-a96e-911c619d1cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.786889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094d5e-ac66-4962-a96e-911c619d1cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:48.787367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1294/2632] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:48.800158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.802124 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:48.827011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-46094d5e-ac66-4962-a96e-911c619d1cc6 None None] MFA Rules not processed for user `c6358240b0084c0bbc475e7db74f74a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:48.828935 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-46094d5e-ac66-4962-a96e-911c619d1cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:48.903620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1f968-1fc3-4b14-a652-7745a4a23897 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:48.917405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1339/2633] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 07 16:05:48.918488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094d5e-ac66-4962-a96e-911c619d1cc6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:48.962146 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:48.974582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:48.976182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:48.976803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:48.976942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:48.977313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.016481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094d5e-ac66-4962-a96e-911c619d1cc6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:49.023587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:49.024278 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.024593 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.025061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.025727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.028391 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.052774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.055724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.066205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1340/2634] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:48 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:49.071409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1295/2635] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:49.079246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:49.182978 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d4023-c88b-4f9c-ac8d-4a925d7330fe admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:49.192287 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:49.192778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.193277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.193621 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.194644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.196695 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.227527 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1341/2636] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:49 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:49.228805 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:49.245573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3dca-5132-4caf-a9a5-f4ed27e9e992 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:49.252458 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:49.253752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.254221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.254540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.255730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.258293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.271379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1296/2637] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:49.275159 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:49.288140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d2a5-7a6c-4450-b591-59eef28c029b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:49.294347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:49.296151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.296671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.297232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] PATH_INFO: `/v3/projects/4d1267fecc0c4718912ddfabacb2c7fb/users/af775f7549ab4554a7f3b1fa8b94e722/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.329086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1267fecc0c4718912ddfabacb2c7fb, user_id=af775f7549ab4554a7f3b1fa8b94e722, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.331933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.358950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1342/2638] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:49 2022] PUT /identity/v3/projects/4d1267fecc0c4718912ddfabacb2c7fb/users/af775f7549ab4554a7f3b1fa8b94e722/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:49.359217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:49.371386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60886724-143f-45a5-84b8-294e7cac1636 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:49.453522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b2180-d234-42bf-b9f3-bb1085d6962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:49.454557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b2180-d234-42bf-b9f3-bb1085d6962e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.454736 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b2180-d234-42bf-b9f3-bb1085d6962e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.455298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b2180-d234-42bf-b9f3-bb1085d6962e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.456380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b2180-d234-42bf-b9f3-bb1085d6962e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.458243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b2180-d234-42bf-b9f3-bb1085d6962e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.473874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1297/2639] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:49.478623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:49.493335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7aee17-10df-48b6-bf85-ef3cbf2b2822 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:49.500230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:49.501313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.501698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.502259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] PATH_INFO: `/v3/projects/4d1267fecc0c4718912ddfabacb2c7fb/users/af775f7549ab4554a7f3b1fa8b94e722/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.504169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1267fecc0c4718912ddfabacb2c7fb, user_id=af775f7549ab4554a7f3b1fa8b94e722, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.506869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.520107 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1343/2640] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:49 2022] PUT /identity/v3/projects/4d1267fecc0c4718912ddfabacb2c7fb/users/af775f7549ab4554a7f3b1fa8b94e722/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:49.531512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.532668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.533798 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.588158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] MFA Rules not processed for user `af775f7549ab4554a7f3b1fa8b94e722`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:49.589748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:49.721502 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1298/2641] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:49.736179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:49.811423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed7c3a-eff5-440c-ad8a-a2f7c288e8dd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:49.815785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:49.816614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:49.816972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:49.817326 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:49.873325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:49.970484 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:49.974473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:49.990811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1344/2642] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:51.292741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:51.294182 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:51.304079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:51.433913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebdbc-bac9-4c93-ac2c-55c6634c72af None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:51.461773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:51.467486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:51.467486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:51.467486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:51.475242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:51.520968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:51.523413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:51.544432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1299/2643] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:54.547781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:54.549320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:54.622713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd36d69-0341-45db-a48d-ad9c164704f4 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:54.637923 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:54.639150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:54.640788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:54.641470 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] PATH_INFO: `/v3/users/5c98ae10605b4af1b367a28870919e47` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:54.642887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c98ae10605b4af1b367a28870919e47)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:54.645160 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:54.701691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c98ae10605b4af1b367a28870919e47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:54.702656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c98ae10605b4af1b367a28870919e47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:54.703257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c98ae10605b4af1b367a28870919e47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:54.711534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c98ae10605b4af1b367a28870919e47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:54.730993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c98ae10605b4af1b367a28870919e47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:54.745523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1345/2644] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:54 2022] DELETE /identity/v3/users/5c98ae10605b4af1b367a28870919e47 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:54.929269 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:54.944118 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afc3a-1db7-4d5b-a34b-55258c8cc32a service placement] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:55.008345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:55.009319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:55.009605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:55.009880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] PATH_INFO: `/v3/projects/8bc2a18d64e440eabb0f4f4a072cf671` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:55.011072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc2a18d64e440eabb0f4f4a072cf671)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:55.013052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:55.064076 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc2a18d64e440eabb0f4f4a072cf671', 'request_id': 'req-c9b32dbf-f7f3-4308-a432-00266370351e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.070478 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc2a18d64e440eabb0f4f4a072cf671', 'request_id': 'req-c9b32dbf-f7f3-4308-a432-00266370351e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.079388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc2a18d64e440eabb0f4f4a072cf671', 'request_id': 'req-c9b32dbf-f7f3-4308-a432-00266370351e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.092416 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] The token cache is being invalidate because project 8bc2a18d64e440eabb0f4f4a072cf671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:55.092610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.093282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.095130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1300/2645] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:54 2022] DELETE /identity/v3/projects/8bc2a18d64e440eabb0f4f4a072cf671 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:55.098743 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:55.100426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:55.115149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e7010-ce60-46a5-9624-50ed59f43423 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:55.181558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:55.206610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:55.207710 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:55.208463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:55.208463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] PATH_INFO: `/v3/users/e929fc58e8e24db8b5ac4f7976b54320` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:55.209754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e929fc58e8e24db8b5ac4f7976b54320)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:55.211784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:55.287850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e929fc58e8e24db8b5ac4f7976b54320'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.289383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e929fc58e8e24db8b5ac4f7976b54320'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.290176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e929fc58e8e24db8b5ac4f7976b54320'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.306304 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e929fc58e8e24db8b5ac4f7976b54320'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.325352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e929fc58e8e24db8b5ac4f7976b54320'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.343886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1346/2646] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:55 2022] DELETE /identity/v3/users/e929fc58e8e24db8b5ac4f7976b54320 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:55.548682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:55.550391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:55.565967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32dbf-f7f3-4308-a432-00266370351e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:55.601067 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:55.644707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:55.645909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:55.646189 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:55.646462 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] PATH_INFO: `/v3/projects/af77fdfaf6cb4ae18d33d3ae4bac9459` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:55.647683 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af77fdfaf6cb4ae18d33d3ae4bac9459)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:55.649622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:55.703810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af77fdfaf6cb4ae18d33d3ae4bac9459', 'request_id': 'req-d393950a-7368-493d-bfc3-8d5d12961f40'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.712231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af77fdfaf6cb4ae18d33d3ae4bac9459', 'request_id': 'req-d393950a-7368-493d-bfc3-8d5d12961f40'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.721936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af77fdfaf6cb4ae18d33d3ae4bac9459', 'request_id': 'req-d393950a-7368-493d-bfc3-8d5d12961f40'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.736018 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] The token cache is being invalidate because project af77fdfaf6cb4ae18d33d3ae4bac9459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:55.736449 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.737275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:55.740049 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1301/2647] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:55 2022] DELETE /identity/v3/projects/af77fdfaf6cb4ae18d33d3ae4bac9459 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:55.905392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:55.908558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:56.009817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8a9a2-9560-48de-a3e4-6b3e733cbc79 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:56.081098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:56.081964 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:56.082314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:56.082659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:56.085174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:56.142288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:56.182161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:56.183615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:56.198965 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1347/2648] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:05:57.011986 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:57.014324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:57.086102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393950a-7368-493d-bfc3-8d5d12961f40 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:57.090775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:57.092568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:57.092958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:57.093249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] PATH_INFO: `/v3/users/b4425a2ff4174303a24d9a816d70e49a` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:57.094539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4425a2ff4174303a24d9a816d70e49a)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:57.096765 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:57.156254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4425a2ff4174303a24d9a816d70e49a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.156950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4425a2ff4174303a24d9a816d70e49a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.164109 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4425a2ff4174303a24d9a816d70e49a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.164881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4425a2ff4174303a24d9a816d70e49a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.167547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:57.181397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4425a2ff4174303a24d9a816d70e49a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.201752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1302/2649] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:57 2022] DELETE /identity/v3/users/b4425a2ff4174303a24d9a816d70e49a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:57.385405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:57.397890 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4f22-585b-442d-84fc-86db4e556298 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:57.460096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:57.461078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:57.461428 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:57.461771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] PATH_INFO: `/v3/projects/816567bd76d84e5f9de58969bf4ebc2c` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:57.463028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=816567bd76d84e5f9de58969bf4ebc2c)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:57.465034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:57.530187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816567bd76d84e5f9de58969bf4ebc2c', 'request_id': 'req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.537329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816567bd76d84e5f9de58969bf4ebc2c', 'request_id': 'req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.544026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816567bd76d84e5f9de58969bf4ebc2c', 'request_id': 'req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.557968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] The token cache is being invalidate because project 816567bd76d84e5f9de58969bf4ebc2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:57.558384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.559516 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.561629 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1348/2650] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:57 2022] DELETE /identity/v3/projects/816567bd76d84e5f9de58969bf4ebc2c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:57.565427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:57.566937 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:57.581044 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe99bef-6192-4075-8352-5c021ec7c0ae admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:57.654995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:57.655896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:57.656174 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:57.656617 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] PATH_INFO: `/v3/users/a3619c1469a64861af3d1dde7ff4ed24` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:57.657820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3619c1469a64861af3d1dde7ff4ed24)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:57.664567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:57.746858 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3619c1469a64861af3d1dde7ff4ed24'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.747649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3619c1469a64861af3d1dde7ff4ed24'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.755823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3619c1469a64861af3d1dde7ff4ed24'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.756775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3619c1469a64861af3d1dde7ff4ed24'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.765315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3619c1469a64861af3d1dde7ff4ed24'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.788022 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:57.789439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:57.793282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1303/2651] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:57 2022] DELETE /identity/v3/users/a3619c1469a64861af3d1dde7ff4ed24 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:57.805130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f17ae7-09e3-4bc3-a588-ee4f5cc239b4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:57.892543 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:57.893554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:57.893887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:57.894165 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] PATH_INFO: `/v3/users/d09f3ab53f494dd0a8a0d02396a6248b` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:57.895731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d09f3ab53f494dd0a8a0d02396a6248b)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:57.897962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:57.900525 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:57.956698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd09f3ab53f494dd0a8a0d02396a6248b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.957540 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd09f3ab53f494dd0a8a0d02396a6248b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.958422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd09f3ab53f494dd0a8a0d02396a6248b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.970506 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd09f3ab53f494dd0a8a0d02396a6248b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:57.989943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d94caa9d-ab86-4b2a-ba4b-a308acaaccc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd09f3ab53f494dd0a8a0d02396a6248b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.001882 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.003699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.019988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1349/2652] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:05:57 2022] DELETE /identity/v3/users/d09f3ab53f494dd0a8a0d02396a6248b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:58.023742 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.023980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.024262 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.026978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1207cb-517d-498c-91a5-d0bc32881d8b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.046147 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:58.047380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.086555 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:58.105446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.106649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.107412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.108525 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] PATH_INFO: `/v3/projects/7f92aa0b9b6d413aa436a2f81fa9eeff` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.110175 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f92aa0b9b6d413aa436a2f81fa9eeff)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.112075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.116949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1350/2653] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:58.127025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.129040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.179877 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f92aa0b9b6d413aa436a2f81fa9eeff', 'request_id': 'req-92518c55-b005-47f3-b5a0-25f80e5c6cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.193804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f92aa0b9b6d413aa436a2f81fa9eeff', 'request_id': 'req-92518c55-b005-47f3-b5a0-25f80e5c6cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.203706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3c098-54d7-4d66-89a0-c92ce15dce5f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.207009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f92aa0b9b6d413aa436a2f81fa9eeff', 'request_id': 'req-92518c55-b005-47f3-b5a0-25f80e5c6cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.222421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] The token cache is being invalidate because project 7f92aa0b9b6d413aa436a2f81fa9eeff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:58.222851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.224178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-92518c55-b005-47f3-b5a0-25f80e5c6cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.226587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1304/2654] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:58 2022] DELETE /identity/v3/projects/7f92aa0b9b6d413aa436a2f81fa9eeff => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:58.231966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e51d-1eef-4daf-9657-70e3154f854e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.232690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e51d-1eef-4daf-9657-70e3154f854e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.233090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e51d-1eef-4daf-9657-70e3154f854e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.244408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:58.252878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5724e51d-1eef-4daf-9657-70e3154f854e None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:58.254160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5724e51d-1eef-4daf-9657-70e3154f854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.264652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.265403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.265691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.265967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.268215 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.324585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1305/2655] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:58.332760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0165c-7174-4823-8016-149583d50370 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.333048 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0165c-7174-4823-8016-149583d50370 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.333327 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0165c-7174-4823-8016-149583d50370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.347984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.349424 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.354062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9cb0165c-7174-4823-8016-149583d50370 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:58.355251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9cb0165c-7174-4823-8016-149583d50370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.360196 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1351/2656] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:58.363923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1306/2657] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:58.379972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.379972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d3a86-55c9-4706-8f39-0953963d0934 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.385820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.385820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.385820 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.387003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.387698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:58.388011 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.390354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.405172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1352/2658] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:58.409857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0165c-7174-4823-8016-149583d50370 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.419853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0165c-7174-4823-8016-149583d50370 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.426404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.427230 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.427537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.428029 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.428757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.431075 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.462533 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1307/2659] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:58.467192 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.478069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c3b8f-27fc-4f66-8f64-052cb9894466 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.541048 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.541732 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.542014 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.542290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.542998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.545062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.566201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1353/2660] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:58.570539 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.581887 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffcdc8-5c43-4b3c-aa08-abf54d7fd2dc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.586801 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.587558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.587846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.588121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.588875 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.590459 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.602097 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1308/2661] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:58.605902 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.617062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738f333-92c0-4edc-b477-954a4a8500f2 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.622050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.622701 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.623007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.623320 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.624392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.625998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.636427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1354/2662] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:58.641692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.654091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.656474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.656664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be55136-caf1-4f94-a22c-f8d3377c8bb0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.661880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8119b1-b960-4a9b-beba-f8c722c3c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.662970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8119b1-b960-4a9b-beba-f8c722c3c406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.663428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8119b1-b960-4a9b-beba-f8c722c3c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.663720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8119b1-b960-4a9b-beba-f8c722c3c406 admin admin] PATH_INFO: `/v3/projects/834edf421ab24f219e9c1fe425d4d1aa/users/e6c5e211cb0945128c8588a3ec0d510d/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.672110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86f50-e907-4d4f-86e1-fac4b974a912 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.677295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.678279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.678665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.679036 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] PATH_INFO: `/v3/projects/c4fa50ed2e644c8c933657803b6a547d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.680476 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4fa50ed2e644c8c933657803b6a547d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.682511 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.683592 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8119b1-b960-4a9b-beba-f8c722c3c406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834edf421ab24f219e9c1fe425d4d1aa, user_id=e6c5e211cb0945128c8588a3ec0d510d, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.688007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8119b1-b960-4a9b-beba-f8c722c3c406 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.709490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1309/2663] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:58 2022] PUT /identity/v3/projects/834edf421ab24f219e9c1fe425d4d1aa/users/e6c5e211cb0945128c8588a3ec0d510d/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:05:58.720748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.721100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.721387 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.741130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:58.744205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4fa50ed2e644c8c933657803b6a547d', 'request_id': 'req-1120797b-e8ed-4d0c-910b-774e66105d17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.752401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] MFA Rules not processed for user `e6c5e211cb0945128c8588a3ec0d510d`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:58.753583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.753972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4fa50ed2e644c8c933657803b6a547d', 'request_id': 'req-1120797b-e8ed-4d0c-910b-774e66105d17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.763069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4fa50ed2e644c8c933657803b6a547d', 'request_id': 'req-1120797b-e8ed-4d0c-910b-774e66105d17'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.764513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:58.778916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] The token cache is being invalidate because project c4fa50ed2e644c8c933657803b6a547d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:05:58.779410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.780677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-1120797b-e8ed-4d0c-910b-774e66105d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:05:58.782827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1355/2664] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:05:58 2022] DELETE /identity/v3/projects/c4fa50ed2e644c8c933657803b6a547d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:58.787361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b1317-29c4-4a9b-bf59-320084c498ff None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.787648 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b1317-29c4-4a9b-bf59-320084c498ff None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.788089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b1317-29c4-4a9b-bf59-320084c498ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.806996 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-799b1317-29c4-4a9b-bf59-320084c498ff None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:58.808183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-799b1317-29c4-4a9b-bf59-320084c498ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.838066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1310/2665] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:58.844713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.844713 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.886737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1356/2666] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:58.894744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347befc3-01e2-456d-94ce-2a864b8569da None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.895121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347befc3-01e2-456d-94ce-2a864b8569da None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.895443 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347befc3-01e2-456d-94ce-2a864b8569da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.914636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-347befc3-01e2-456d-94ce-2a864b8569da None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:58.915821 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-347befc3-01e2-456d-94ce-2a864b8569da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:58.921672 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470dbe-8f11-481a-90b6-21387f31b212 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.924442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1357/2667] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:58.926870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.927650 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.927939 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.928205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.928825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347befc3-01e2-456d-94ce-2a864b8569da None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.928969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.931037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.940052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347befc3-01e2-456d-94ce-2a864b8569da None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:58.947083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:58.947764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:58.948042 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:58.948316 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:58.949130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:05:58.949425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:58.951019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:58.965271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1311/2668] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:58.966417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.968203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1358/2669] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:05:58.972098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:58.982015 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f355ce-20e6-4b38-85a6-1138108c81eb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.033282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3917d6-4c77-43a6-bea0-728f4475fb7d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.037499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.037826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.038164 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.038454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.038580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.038731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.038852 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.039016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.039423 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.039691 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.041357 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.041622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.064153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1312/2670] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:59.068035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.070615 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1359/2671] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:05:58 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:05:59.075150 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.077794 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11a295-7502-43a0-9467-84f4f3a421d5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.102733 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:59.120941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:59.138744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.139784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.140077 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.140375 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.141107 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.141939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67d637-ae83-47e0-8fed-532d99cbc304 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.142702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.147047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.147735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.148003 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.148275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.148995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.150913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.159086 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1313/2672] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:59.163234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.173226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1360/2673] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:05:59 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:05:59.174698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82dcae-9d19-4546-8e4b-52e116b69f40 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.179149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.180173 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.181088 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.181369 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.181642 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] PATH_INFO: `/v3/projects/ad78adf795b641d6a4a5ff380e32e58d/users/6768bf3251e74e8b9c35111294f1b367/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.190461 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb0d71-746f-4bc7-b7e5-cdd2bfa7cb6e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.195794 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.196555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.196953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.197236 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.197963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.199406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad78adf795b641d6a4a5ff380e32e58d, user_id=6768bf3251e74e8b9c35111294f1b367, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.199693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.202222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.211193 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1361/2674] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:59.215181 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.216222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1314/2675] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:59 2022] PUT /identity/v3/projects/ad78adf795b641d6a4a5ff380e32e58d/users/6768bf3251e74e8b9c35111294f1b367/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:59.219878 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.225463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717feb0c-5fca-4b6b-bc8f-61461e69aafc admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.283214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.283452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d48955-9567-4759-9211-c5955cdd1b9a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.283877 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.284156 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.284458 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.285178 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.286786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.288553 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.289637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.289997 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.290383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.291214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.292891 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.297431 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1362/2676] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:05:59.301483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.307024 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1315/2677] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:05:59.308638 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.312674 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd6e5d-0a9a-458e-a7a5-001e1779754e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.317547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3eb591-2e71-4be6-82f5-4987256b39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.318448 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb591-2e71-4be6-82f5-4987256b39f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.318729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb591-2e71-4be6-82f5-4987256b39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.319005 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb591-2e71-4be6-82f5-4987256b39f7 admin admin] PATH_INFO: `/v3/projects/b9ba8278c75c4b2e9211d6d924ddb754/users/2fd2fe7bf6f649228aafa233285235f2/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.319116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917ab21-9af9-4696-aedf-9e728aa3acbc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.323741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67605f05-4ab8-48b4-ac05-0a4560dbb7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.324645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67605f05-4ab8-48b4-ac05-0a4560dbb7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.324928 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67605f05-4ab8-48b4-ac05-0a4560dbb7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.325207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67605f05-4ab8-48b4-ac05-0a4560dbb7c7 admin admin] PATH_INFO: `/v3/projects/ad78adf795b641d6a4a5ff380e32e58d/users/6768bf3251e74e8b9c35111294f1b367/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.327787 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67605f05-4ab8-48b4-ac05-0a4560dbb7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad78adf795b641d6a4a5ff380e32e58d, user_id=6768bf3251e74e8b9c35111294f1b367, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.330572 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67605f05-4ab8-48b4-ac05-0a4560dbb7c7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.334426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3eb591-2e71-4be6-82f5-4987256b39f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9ba8278c75c4b2e9211d6d924ddb754, user_id=2fd2fe7bf6f649228aafa233285235f2, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.337098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3eb591-2e71-4be6-82f5-4987256b39f7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.341114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1316/2678] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:59 2022] PUT /identity/v3/projects/ad78adf795b641d6a4a5ff380e32e58d/users/6768bf3251e74e8b9c35111294f1b367/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:59.350250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1363/2679] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:05:59 2022] PUT /identity/v3/projects/b9ba8278c75c4b2e9211d6d924ddb754/users/2fd2fe7bf6f649228aafa233285235f2/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:05:59.362299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.362586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.363052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.393908 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] MFA Rules not processed for user `2fd2fe7bf6f649228aafa233285235f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:59.395138 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:59.410425 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.410969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.410969 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.437952 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] MFA Rules not processed for user `6768bf3251e74e8b9c35111294f1b367`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:05:59.439151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:59.462775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1317/2680] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:05:59.473773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.475142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:59.488307 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:59.511360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1364/2681] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:05:59.549231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c60729-22bc-45d3-a22e-dc0189080803 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.549231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.549231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.549231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.549693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.549968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:05:59.551338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:05:59.555127 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.556574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.563734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bfa15-4774-48f7-8f20-48db5c1f2247 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:05:59.566066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1318/2682] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:05:59.567623 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:05:59.568261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:05:59.568583 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:05:59.568860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:05:59.582970 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:05:59.634914 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:05:59.636422 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:05:59.649143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1365/2683] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:05:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:06:01.567060 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:01.568619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:01.631281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d9b1a-60bb-432e-ac87-b24140abe047 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:01.635047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240d30e-9d6a-4ee3-b7fd-41608a32df2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:01.635722 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d30e-9d6a-4ee3-b7fd-41608a32df2b service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:01.636006 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d30e-9d6a-4ee3-b7fd-41608a32df2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:01.636345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d30e-9d6a-4ee3-b7fd-41608a32df2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:01.641054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0240d30e-9d6a-4ee3-b7fd-41608a32df2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:01.708126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240d30e-9d6a-4ee3-b7fd-41608a32df2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:01.709703 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240d30e-9d6a-4ee3-b7fd-41608a32df2b service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:01.720273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1319/2684] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:02.265072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:02.334878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624898e9-e488-4ea3-b5a1-d061a96860a2 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:02.339576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:02.340587 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:02.340865 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:02.341138 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] PATH_INFO: `/v3/users/e6c5e211cb0945128c8588a3ec0d510d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:02.342405 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c5e211cb0945128c8588a3ec0d510d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:02.344319 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:02.396745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c5e211cb0945128c8588a3ec0d510d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:02.397441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c5e211cb0945128c8588a3ec0d510d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:02.398201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c5e211cb0945128c8588a3ec0d510d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:02.406024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c5e211cb0945128c8588a3ec0d510d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:02.418284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c5e211cb0945128c8588a3ec0d510d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:02.433951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1366/2685] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:02 2022] DELETE /identity/v3/users/e6c5e211cb0945128c8588a3ec0d510d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:02.438850 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:02.439154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:02.439428 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:02.457147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:02.458363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:02.522237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1320/2686] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:02.532132 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:02.533581 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:02.575045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:02.595905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642785af-86cb-459f-871e-82283a2fdc1b admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:02.599585 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:02.600303 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:02.600606 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:02.600886 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:02.611302 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:02.612806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:02.623739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1367/2687] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:03.083926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:03.085490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:03.094028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:03.098388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a07a6-877f-487f-9593-0ec639a0564a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:03.104500 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:03.105446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:03.105807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:03.106156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] PATH_INFO: `/v3/projects/834edf421ab24f219e9c1fe425d4d1aa` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:03.108154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834edf421ab24f219e9c1fe425d4d1aa)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:03.110178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:03.172832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834edf421ab24f219e9c1fe425d4d1aa', 'request_id': 'req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.179864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834edf421ab24f219e9c1fe425d4d1aa', 'request_id': 'req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.189130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834edf421ab24f219e9c1fe425d4d1aa', 'request_id': 'req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.199639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] The token cache is being invalidate because project 834edf421ab24f219e9c1fe425d4d1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:03.199923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.200594 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.202418 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1321/2688] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:03 2022] DELETE /identity/v3/projects/834edf421ab24f219e9c1fe425d4d1aa => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:03.206437 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:03.216043 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81035567-e124-4116-abc9-60c47f0028c6 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:03.275194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:03.276082 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:03.276386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:03.276706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] PATH_INFO: `/v3/users/6768bf3251e74e8b9c35111294f1b367` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:03.277926 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6768bf3251e74e8b9c35111294f1b367)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:03.280539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:03.329829 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6768bf3251e74e8b9c35111294f1b367'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.330380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6768bf3251e74e8b9c35111294f1b367'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.331007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6768bf3251e74e8b9c35111294f1b367'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.338183 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6768bf3251e74e8b9c35111294f1b367'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.351966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-773f24f6-1f27-4cb8-9035-6f9017f5cf1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6768bf3251e74e8b9c35111294f1b367'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.368237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1368/2689] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:03 2022] DELETE /identity/v3/users/6768bf3251e74e8b9c35111294f1b367 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:03.661031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:03.662897 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:03.678111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6dcb5-9cb5-4915-954c-036ec4ac86c6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:03.745272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:03.746292 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:03.746676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:03.747055 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] PATH_INFO: `/v3/projects/ad78adf795b641d6a4a5ff380e32e58d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:03.748621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad78adf795b641d6a4a5ff380e32e58d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:03.750628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:03.809104 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad78adf795b641d6a4a5ff380e32e58d', 'request_id': 'req-36241299-b2ca-4873-80d1-5b1f87748f27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.815567 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad78adf795b641d6a4a5ff380e32e58d', 'request_id': 'req-36241299-b2ca-4873-80d1-5b1f87748f27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.823659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad78adf795b641d6a4a5ff380e32e58d', 'request_id': 'req-36241299-b2ca-4873-80d1-5b1f87748f27'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.834898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] The token cache is being invalidate because project ad78adf795b641d6a4a5ff380e32e58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:03.835374 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.836020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-36241299-b2ca-4873-80d1-5b1f87748f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:03.837618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1322/2690] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:03 2022] DELETE /identity/v3/projects/ad78adf795b641d6a4a5ff380e32e58d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:03.843564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f213e-07ab-4668-8cb9-00c87445fe00 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:03.843858 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f213e-07ab-4668-8cb9-00c87445fe00 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:03.844140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f213e-07ab-4668-8cb9-00c87445fe00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:03.862647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-0f8f213e-07ab-4668-8cb9-00c87445fe00 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:03.863770 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0f8f213e-07ab-4668-8cb9-00c87445fe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:03.919125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:03.930128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1369/2691] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:03.938052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0501-9199-470f-9c5f-642b8806c689 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:03.938345 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0501-9199-470f-9c5f-642b8806c689 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:03.938610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0501-9199-470f-9c5f-642b8806c689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:03.956649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-601b0501-9199-470f-9c5f-642b8806c689 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:03.957848 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-601b0501-9199-470f-9c5f-642b8806c689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:03.971141 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1323/2692] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:03.972442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f213e-07ab-4668-8cb9-00c87445fe00 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:03.983012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f213e-07ab-4668-8cb9-00c87445fe00 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:03.987980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:03.989250 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:03.989576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:03.989917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:03.990738 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:06:03.991044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:03.992825 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.005404 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1370/2693] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:06:04.009602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b0501-9199-470f-9c5f-642b8806c689 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.019093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b0501-9199-470f-9c5f-642b8806c689 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.024867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.025558 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.025840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.026116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.026807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.028759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.037866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:04.061317 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1324/2694] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:04 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:04.065558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.077764 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11780387-cfee-477e-b595-3e75ca6fcfc1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.151116 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.151872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.152162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.152464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.153179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.156261 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.176446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1371/2695] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:04 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:04.180450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.194913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684b16d-a339-4732-9cb5-5687dd5e69f0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.199540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.200207 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.200519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.200797 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.201587 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.203184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.214231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1325/2696] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:04.219266 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.229801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3097-9a0c-4477-ad8c-ecbd1bf1b245 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.235436 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2c61c-f010-4949-8628-ffabd74e11ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.236130 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2c61c-f010-4949-8628-ffabd74e11ae admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.236445 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2c61c-f010-4949-8628-ffabd74e11ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.236724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2c61c-f010-4949-8628-ffabd74e11ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.237450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2c61c-f010-4949-8628-ffabd74e11ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.239047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2c61c-f010-4949-8628-ffabd74e11ae admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.250189 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1372/2697] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:04.254376 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.264759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc3c36-dec2-4783-95e2-7b4df6369b88 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.269485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.270363 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.270653 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.270931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] PATH_INFO: `/v3/projects/8229792d46e54e28b79208db9b611451/users/9d60eca9c7f040ba896aa628ec10dbbd/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.286504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8229792d46e54e28b79208db9b611451, user_id=9d60eca9c7f040ba896aa628ec10dbbd, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.289300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.303615 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1326/2698] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:04 2022] PUT /identity/v3/projects/8229792d46e54e28b79208db9b611451/users/9d60eca9c7f040ba896aa628ec10dbbd/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:04.315378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.315930 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.316114 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.351805 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] MFA Rules not processed for user `9d60eca9c7f040ba896aa628ec10dbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:04.353092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:04.443121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1373/2699] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:04.455698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.457472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:04.524176 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275d82f-8d69-478e-b9cc-ffb918f1cd4b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.527996 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0206e09e4f426d9f4b54543d89c039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.528709 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.528962 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.529884 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.552485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.554482 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.567091 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1327/2700] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:04.790153 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.791778 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:04.863251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:04.881259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1304e-7528-4500-bcd0-2f56258eca24 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.889381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.890414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.890699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.891023 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] PATH_INFO: `/v3/users/2fd2fe7bf6f649228aafa233285235f2` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.892471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd2fe7bf6f649228aafa233285235f2)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.894963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.950404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:04.952218 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:04.958398 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd2fe7bf6f649228aafa233285235f2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:04.959131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd2fe7bf6f649228aafa233285235f2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:04.959922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd2fe7bf6f649228aafa233285235f2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:04.968910 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a909a1-8790-46f8-90d2-30fa6781d3e2 service glance] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:04.970663 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd2fe7bf6f649228aafa233285235f2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:04.975425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:04.976420 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:04.976747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:04.977047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] PATH_INFO: `/v3/users/c16cee31cb334a84aa14482bbb494664` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:04.979062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16cee31cb334a84aa14482bbb494664)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:04.981043 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:04.988800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8619bf08-6bf6-4d31-a859-7c8f0acc914e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd2fe7bf6f649228aafa233285235f2'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.006586 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1374/2701] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:04 2022] DELETE /identity/v3/users/2fd2fe7bf6f649228aafa233285235f2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:05.010977 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.011637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.011637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.041838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:05.043050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:05.066862 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16cee31cb334a84aa14482bbb494664'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.068083 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16cee31cb334a84aa14482bbb494664'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.081616 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16cee31cb334a84aa14482bbb494664'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.082340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16cee31cb334a84aa14482bbb494664'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.091358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8c1b3a50-9bf9-4f8e-85d4-b5b855948828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16cee31cb334a84aa14482bbb494664'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.119383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1328/2702] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:04 2022] DELETE /identity/v3/users/c16cee31cb334a84aa14482bbb494664 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:05.122665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1375/2703] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:05.125185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11edab1-2e78-4d8e-a092-4e4529d2af0b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.125724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11edab1-2e78-4d8e-a092-4e4529d2af0b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.126026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11edab1-2e78-4d8e-a092-4e4529d2af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.134619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:05.134619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:05.150608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f11edab1-2e78-4d8e-a092-4e4529d2af0b None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:05.150608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f11edab1-2e78-4d8e-a092-4e4529d2af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:05.175735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:05.204556 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f0f2f-92e0-431b-9339-d7f21ed3497f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:05.217415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1329/2704] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:05.227497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11edab1-2e78-4d8e-a092-4e4529d2af0b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:05.236315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:05.263463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:05.264912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.265205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.265481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.275659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:05.277097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:05.285866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11edab1-2e78-4d8e-a092-4e4529d2af0b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:05.287861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1376/2705] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:05.290467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:05.291132 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.291415 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.291692 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.304540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:05.305988 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:05.317126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1330/2706] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:05.746468 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:05.752180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:05.776770 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:05.778390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:05.782881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903003f-ef1d-4483-8e94-faa2e0823e58 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:05.794329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:05.796286 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.797091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.797447 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] PATH_INFO: `/v3/projects/b9ba8278c75c4b2e9211d6d924ddb754` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.798609 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805de5-a0dd-4457-8860-14daa1291855 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:05.798984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9ba8278c75c4b2e9211d6d924ddb754)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:05.801168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:05.803724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:05.804955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.805246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.805566 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] PATH_INFO: `/v3/projects/7a5590f870754829924beac551ba3742` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.806892 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5590f870754829924beac551ba3742)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:05.808865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:05.853808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9ba8278c75c4b2e9211d6d924ddb754', 'request_id': 'req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.860609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9ba8278c75c4b2e9211d6d924ddb754', 'request_id': 'req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.862142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5590f870754829924beac551ba3742', 'request_id': 'req-72e224c8-d99f-4aff-a836-47a6d3d6fc90'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.866933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9ba8278c75c4b2e9211d6d924ddb754', 'request_id': 'req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.869089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5590f870754829924beac551ba3742', 'request_id': 'req-72e224c8-d99f-4aff-a836-47a6d3d6fc90'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.878156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5590f870754829924beac551ba3742', 'request_id': 'req-72e224c8-d99f-4aff-a836-47a6d3d6fc90'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.880601 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] The token cache is being invalidate because project b9ba8278c75c4b2e9211d6d924ddb754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:05.880895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.881537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-022f1584-c0fa-4c75-a4d1-f44fceb1e4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.883077 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1377/2707] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:05 2022] DELETE /identity/v3/projects/b9ba8278c75c4b2e9211d6d924ddb754 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:05.889549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597da0f-c304-477f-9aae-74ae6bc34e72 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.889800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597da0f-c304-477f-9aae-74ae6bc34e72 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.890080 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597da0f-c304-477f-9aae-74ae6bc34e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.892515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] The token cache is being invalidate because project 7a5590f870754829924beac551ba3742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:05.892808 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.893447 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72e224c8-d99f-4aff-a836-47a6d3d6fc90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:05.894955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1331/2708] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:05 2022] DELETE /identity/v3/projects/7a5590f870754829924beac551ba3742 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:05.899845 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef257a-fc30-4bdd-b2af-d525bb88f631 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.900162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef257a-fc30-4bdd-b2af-d525bb88f631 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.900494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef257a-fc30-4bdd-b2af-d525bb88f631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:05.908434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d597da0f-c304-477f-9aae-74ae6bc34e72 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:05.909636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d597da0f-c304-477f-9aae-74ae6bc34e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:05.919325 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cbef257a-fc30-4bdd-b2af-d525bb88f631 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:05.921211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cbef257a-fc30-4bdd-b2af-d525bb88f631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:05.974297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:05.983079 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1378/2709] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:05.993350 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1332/2710] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:05.999110 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73b32-4d47-469d-814c-a2432f3b1c0d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:05.999613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73b32-4d47-469d-814c-a2432f3b1c0d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:05.999766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73b32-4d47-469d-814c-a2432f3b1c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.020084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-16b73b32-4d47-469d-814c-a2432f3b1c0d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:06.021657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-16b73b32-4d47-469d-814c-a2432f3b1c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:06.040713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1379/2711] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:06.046723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef257a-fc30-4bdd-b2af-d525bb88f631 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.049345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa01c0-e215-40a6-8ab5-67a3decd3a23 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.049627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa01c0-e215-40a6-8ab5-67a3decd3a23 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.049905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa01c0-e215-40a6-8ab5-67a3decd3a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.057430 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef257a-fc30-4bdd-b2af-d525bb88f631 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.062773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.063657 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.063941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.064217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.065103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:06:06.065470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.067100 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.071288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-76aa01c0-e215-40a6-8ab5-67a3decd3a23 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:06.072539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-76aa01c0-e215-40a6-8ab5-67a3decd3a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:06.079978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1333/2712] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:06:06.082571 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1380/2713] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:06.085984 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.086499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa01c0-e215-40a6-8ab5-67a3decd3a23 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.095047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c53bb-989a-4ea2-9aee-2d649dcd3654 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.097900 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa01c0-e215-40a6-8ab5-67a3decd3a23 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.101271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.101938 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.102324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.102620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.102899 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.103408 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.103536 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.103817 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.104085 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.104963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:06:06.105267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.105386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.107084 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.120065 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1381/2714] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 16:06:06.124669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.140085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1334/2715] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:06 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:06.143874 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.147993 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d963c-d620-44ef-81b3-4015f9a06a69 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.182637 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:06.214297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f092d7-a2b3-4083-87b8-0494c503d2ea admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.217896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.218562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.218852 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.219125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.219320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.219838 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.219995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.220282 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.220610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.221321 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.221786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.223212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.243526 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1335/2716] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:06 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:06.247466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.252212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1382/2717] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:06 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:06.253385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.257980 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a670ce-403e-42b8-bdc3-80b609b13e6c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.316419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.317089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.317372 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.317644 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.318378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.319991 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.320602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed0c93-7d92-44b0-9e49-85ae431b4481 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.325408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.326685 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.326917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.327351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.328115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.330201 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.334154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1336/2718] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:06:06.335586 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.346026 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd274-17f6-491b-8e68-cd1f8c7afa42 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.350876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.351538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.351832 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.352106 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.352853 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.354427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.355012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1383/2719] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:06 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:06.359359 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.367986 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1337/2720] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:06.369008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718158bd-9d74-4e6e-b93f-d6c5f8e7e16e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.372079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.374592 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.375366 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.375653 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.375929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.376745 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.378430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.383066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee752d4-9fa5-4538-bd10-9b29e2bb4c75 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.387773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.389256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.389537 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.389713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1384/2721] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:06.390039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] PATH_INFO: `/v3/projects/0b137cb155a44f5e942abe6bfaa53884/users/8feb891b1be946398e7c35cc34a8f150/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.393834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.409729 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f160f-4b9a-4773-aa0f-4168dd89e3a3 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.411256 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b137cb155a44f5e942abe6bfaa53884, user_id=8feb891b1be946398e7c35cc34a8f150, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.415332 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bcd96-cd31-4a89-b4a8-a3511e473085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.415886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.416097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bcd96-cd31-4a89-b4a8-a3511e473085 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.416275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bcd96-cd31-4a89-b4a8-a3511e473085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.416652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bcd96-cd31-4a89-b4a8-a3511e473085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.417699 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bcd96-cd31-4a89-b4a8-a3511e473085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.420450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bcd96-cd31-4a89-b4a8-a3511e473085 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.431340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1338/2722] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:06 2022] PUT /identity/v3/projects/0b137cb155a44f5e942abe6bfaa53884/users/8feb891b1be946398e7c35cc34a8f150/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:06.433317 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1385/2723] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:06:06.437582 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.441523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.441816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.442098 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.453012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:06.473920 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] MFA Rules not processed for user `8feb891b1be946398e7c35cc34a8f150`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:06.475277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:06.506823 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c02d6-d2dc-4842-8b02-df6e146a9eeb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.512032 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310956a-f60f-4998-90d1-0156a37a4ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.513017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310956a-f60f-4998-90d1-0156a37a4ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.513303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310956a-f60f-4998-90d1-0156a37a4ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.514156 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310956a-f60f-4998-90d1-0156a37a4ccd admin admin] PATH_INFO: `/v3/projects/aad305bb9275482b95b858fa3669e8e9/users/4244e60b4d424bf29575b8214aadde9a/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.528588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310956a-f60f-4998-90d1-0156a37a4ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad305bb9275482b95b858fa3669e8e9, user_id=4244e60b4d424bf29575b8214aadde9a, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.531480 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310956a-f60f-4998-90d1-0156a37a4ccd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.546550 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1339/2724] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:06 2022] PUT /identity/v3/projects/aad305bb9275482b95b858fa3669e8e9/users/4244e60b4d424bf29575b8214aadde9a/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:06.550052 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1386/2725] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:06.551593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.551880 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.552157 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.561021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.563134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:06.569608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:06.570782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:06.577613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:06.627409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec9e6-2e7c-4a3e-9452-020f31a9f787 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.635802 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1340/2726] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:06.646422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:06.647819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:06.682552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.683207 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.683489 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.683765 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.708952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe30140-c298-4cf8-9954-911857b35bd5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:06.712750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a0e26-9085-4e26-9ae6-fcf91b1970e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:06.713409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a0e26-9085-4e26-9ae6-fcf91b1970e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:06.713694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a0e26-9085-4e26-9ae6-fcf91b1970e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:06.713972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a0e26-9085-4e26-9ae6-fcf91b1970e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:06.724134 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a0e26-9085-4e26-9ae6-fcf91b1970e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.725613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a0e26-9085-4e26-9ae6-fcf91b1970e8 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.736413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1341/2727] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:06:06.749230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:06.750673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:06.760024 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1387/2728] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:06:07.733411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b8685-74e2-4e6b-a9ea-ae772b387144 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:07.734410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b8685-74e2-4e6b-a9ea-ae772b387144 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:07.734690 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b8685-74e2-4e6b-a9ea-ae772b387144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:07.765515 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8e7b8685-74e2-4e6b-a9ea-ae772b387144 None None] MFA Rules not processed for user `4244e60b4d424bf29575b8214aadde9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:07.766719 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e7b8685-74e2-4e6b-a9ea-ae772b387144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:07.792973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:07.836793 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1342/2729] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:07.847553 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:07.856232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981981f2-c250-4e21-b62b-415d500f2426 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:07.860483 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:07.861152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:07.861434 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:07.861705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:07.870073 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:07.871479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:07.881955 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1388/2730] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:11.343512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e47c-d844-4828-90ea-93cd1bef9287 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:11.343968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e47c-d844-4828-90ea-93cd1bef9287 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:11.344117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e47c-d844-4828-90ea-93cd1bef9287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:11.368785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f132e47c-d844-4828-90ea-93cd1bef9287 None None] MFA Rules not processed for user `4244e60b4d424bf29575b8214aadde9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:11.370702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f132e47c-d844-4828-90ea-93cd1bef9287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:11.380975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1343/2731] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:17.877300 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=69 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:21.359358 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=18 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:28.866676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:28.868121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:28.882341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee027df-61ce-4c39-8aff-e1bf9a1d5f40 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:28.887089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:28.887987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:28.888267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:28.888577 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] PATH_INFO: `/v3/users/c6358240b0084c0bbc475e7db74f74a0` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:28.890046 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6358240b0084c0bbc475e7db74f74a0)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:28.891903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:28.952819 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6358240b0084c0bbc475e7db74f74a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:28.953510 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6358240b0084c0bbc475e7db74f74a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:28.954175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6358240b0084c0bbc475e7db74f74a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:28.963486 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6358240b0084c0bbc475e7db74f74a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:28.983026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6358240b0084c0bbc475e7db74f74a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:28.998341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1389/2732] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:28 2022] DELETE /identity/v3/users/c6358240b0084c0bbc475e7db74f74a0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:29.002914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:29.003333 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:29.003630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:29.021990 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:29.023169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:29.085767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1344/2733] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:29.095141 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:29.096598 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:29.175775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733beb3e-1d0b-4f2a-a872-ff9b00bbf4c5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:29.180439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:29.181612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:29.182557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:29.182987 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:29.195134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:29.197380 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:29.220946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1390/2734] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:29.576664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:29.578117 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:29.591969 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c9d85-ee77-4e7b-9cd5-eeda8010d3ca None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:29.596772 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:29.597636 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:29.597921 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:29.598220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] PATH_INFO: `/v3/projects/a9a04292be0743af8c9acbeb20441416` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:29.599517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a04292be0743af8c9acbeb20441416)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:29.601484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:29.648320 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a04292be0743af8c9acbeb20441416', 'request_id': 'req-c8f62e73-afb8-4b52-bf08-fac5e37fef76'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.654319 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a04292be0743af8c9acbeb20441416', 'request_id': 'req-c8f62e73-afb8-4b52-bf08-fac5e37fef76'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.661645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a04292be0743af8c9acbeb20441416', 'request_id': 'req-c8f62e73-afb8-4b52-bf08-fac5e37fef76'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.671895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] The token cache is being invalidate because project a9a04292be0743af8c9acbeb20441416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:29.672184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.672846 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.674373 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1345/2735] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:29 2022] DELETE /identity/v3/projects/a9a04292be0743af8c9acbeb20441416 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:29.678908 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:29.688602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4da66e-75dd-4101-ae29-b36b5edaf681 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:29.745758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:29.746545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:29.746832 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:29.747105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] PATH_INFO: `/v3/users/af775f7549ab4554a7f3b1fa8b94e722` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:29.748282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af775f7549ab4554a7f3b1fa8b94e722)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:29.750228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:29.765938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:29.794430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af775f7549ab4554a7f3b1fa8b94e722'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.795078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af775f7549ab4554a7f3b1fa8b94e722'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.795684 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af775f7549ab4554a7f3b1fa8b94e722'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.803027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af775f7549ab4554a7f3b1fa8b94e722'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.814644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-08b947f0-9613-4b0c-9e80-bcb2c659d4a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af775f7549ab4554a7f3b1fa8b94e722'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:29.827337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1391/2736] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:29 2022] DELETE /identity/v3/users/af775f7549ab4554a7f3b1fa8b94e722 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:30.090201 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.091648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:30.107676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f62e73-afb8-4b52-bf08-fac5e37fef76 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.168309 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.169276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.169562 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.169836 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] PATH_INFO: `/v3/projects/4d1267fecc0c4718912ddfabacb2c7fb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.171000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1267fecc0c4718912ddfabacb2c7fb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.172941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.202745 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:30.220760 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1267fecc0c4718912ddfabacb2c7fb', 'request_id': 'req-90676ad3-28cd-414b-bd03-43cb9d2adc55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:30.226847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1267fecc0c4718912ddfabacb2c7fb', 'request_id': 'req-90676ad3-28cd-414b-bd03-43cb9d2adc55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:30.233775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1267fecc0c4718912ddfabacb2c7fb', 'request_id': 'req-90676ad3-28cd-414b-bd03-43cb9d2adc55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:30.243775 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] The token cache is being invalidate because project 4d1267fecc0c4718912ddfabacb2c7fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:30.244059 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:30.244697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-90676ad3-28cd-414b-bd03-43cb9d2adc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:30.246211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1346/2737] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:30 2022] DELETE /identity/v3/projects/4d1267fecc0c4718912ddfabacb2c7fb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:30.250984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998591a9-76e0-4364-ba91-ec4a3c0ebabc None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.251379 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998591a9-76e0-4364-ba91-ec4a3c0ebabc None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.251545 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998591a9-76e0-4364-ba91-ec4a3c0ebabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.271047 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-998591a9-76e0-4364-ba91-ec4a3c0ebabc None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:30.272330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-998591a9-76e0-4364-ba91-ec4a3c0ebabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:30.338906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1392/2738] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:30.346961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad4c2-04a8-437b-a360-a11460cccc57 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.347214 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad4c2-04a8-437b-a360-a11460cccc57 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.347491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad4c2-04a8-437b-a360-a11460cccc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.370386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cd1ad4c2-04a8-437b-a360-a11460cccc57 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:30.371588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cd1ad4c2-04a8-437b-a360-a11460cccc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:30.389370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1347/2739] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:30.393578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998591a9-76e0-4364-ba91-ec4a3c0ebabc None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.404140 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998591a9-76e0-4364-ba91-ec4a3c0ebabc None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.409767 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.410377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.411552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.411842 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.412617 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:06:30.412924 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.414550 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.429284 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1393/2740] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:06:30.433840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ad4c2-04a8-437b-a360-a11460cccc57 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.445917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ad4c2-04a8-437b-a360-a11460cccc57 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.450667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.451410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.451812 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.452169 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.452973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.454968 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.486257 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1348/2741] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:30 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:30.490798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.502906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d6cd9-f6f5-48e6-b0d4-3e1269abfc0f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.570687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.571365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.571646 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.571927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.572649 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.574568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.594688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1394/2742] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:30 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:30.599678 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.610116 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263250a6-85de-459f-a207-0318b7dc7b2f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.615584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.616233 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.616552 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.616830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.617540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.619115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.630103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1349/2743] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:30.634547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.644562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1606c6-1da6-4fc1-ab91-01ea08f4395d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.649360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29919c8-bf38-4670-b5d2-073fe7d68003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.650010 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29919c8-bf38-4670-b5d2-073fe7d68003 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.650292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29919c8-bf38-4670-b5d2-073fe7d68003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.650568 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29919c8-bf38-4670-b5d2-073fe7d68003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.651289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29919c8-bf38-4670-b5d2-073fe7d68003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.652919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29919c8-bf38-4670-b5d2-073fe7d68003 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.658069 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:30.666811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1395/2744] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:06:30.669255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.680258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209868f-98f8-47df-a585-ec6b28f0e32b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.685016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.685894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.686178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.686454 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] PATH_INFO: `/v3/projects/e3d1df6ad615404d96ecd12eb7fe96ca/users/d5818087c86e42edbd78eed19706bcda/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.701311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d1df6ad615404d96ecd12eb7fe96ca, user_id=d5818087c86e42edbd78eed19706bcda, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.703923 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.716739 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1350/2745] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:30 2022] PUT /identity/v3/projects/e3d1df6ad615404d96ecd12eb7fe96ca/users/d5818087c86e42edbd78eed19706bcda/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:30.729321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b17631-4535-499e-898d-4513f763b7f8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.729610 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b17631-4535-499e-898d-4513f763b7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.729885 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b17631-4535-499e-898d-4513f763b7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.756313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d5b17631-4535-499e-898d-4513f763b7f8 None None] MFA Rules not processed for user `d5818087c86e42edbd78eed19706bcda`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:30.757557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d5b17631-4535-499e-898d-4513f763b7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:30.821246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1396/2746] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:30.825795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.887680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49befcb1-4106-4881-8abf-1791a4250bf0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:30.892326 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:30.893009 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:30.893293 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:30.893565 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:30.894254 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:30.896648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:30.925508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1351/2747] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:30 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:30.930200 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b17631-4535-499e-898d-4513f763b7f8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:30.941681 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b17631-4535-499e-898d-4513f763b7f8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:31.003106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:31.003803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.004064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.004384 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.005112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:31.007125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:31.027050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1397/2748] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:30 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:31.031033 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:31.042203 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751672a3-40b0-4c2d-a2d2-3699b283509c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:31.046699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:31.047341 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.047619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.047889 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.048635 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:31.050234 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:31.061412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1352/2749] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:31.065414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:31.076000 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3ee9d-40cb-4a73-a311-9940656c7c19 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:31.080549 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:31.081416 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.081696 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.081968 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] PATH_INFO: `/v3/projects/4b028eda9a194c37b6eb3f0d4ad710a9/users/e207ef589b8744768eee9dc96918fedb/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.097067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b028eda9a194c37b6eb3f0d4ad710a9, user_id=e207ef589b8744768eee9dc96918fedb, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:31.099783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:31.112321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1398/2750] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:31 2022] PUT /identity/v3/projects/4b028eda9a194c37b6eb3f0d4ad710a9/users/e207ef589b8744768eee9dc96918fedb/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:31.116759 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:31.126710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42920-8b0b-4014-8413-dc63bb051783 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:31.147120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:31.185485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eee9c-4909-4ae2-8d8c-074e2a97a772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:31.186148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eee9c-4909-4ae2-8d8c-074e2a97a772 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.186426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eee9c-4909-4ae2-8d8c-074e2a97a772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.186732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eee9c-4909-4ae2-8d8c-074e2a97a772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.187442 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eee9c-4909-4ae2-8d8c-074e2a97a772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:31.189081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eee9c-4909-4ae2-8d8c-074e2a97a772 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:31.199411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1353/2751] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:31.203528 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:31.214122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785674c-9863-4ed4-ad1e-59a7c6f64f29 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:31.218749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:31.219616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.219896 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.220171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] PATH_INFO: `/v3/projects/4b028eda9a194c37b6eb3f0d4ad710a9/users/e207ef589b8744768eee9dc96918fedb/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.222172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b028eda9a194c37b6eb3f0d4ad710a9, user_id=e207ef589b8744768eee9dc96918fedb, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:31.225251 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:31.235460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1399/2752] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:31 2022] PUT /identity/v3/projects/4b028eda9a194c37b6eb3f0d4ad710a9/users/e207ef589b8744768eee9dc96918fedb/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:31.246913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.247198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.247477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.274082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] MFA Rules not processed for user `e207ef589b8744768eee9dc96918fedb`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:31.275280 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:31.343241 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1354/2753] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:31.354800 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:31.356628 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:31.405781 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:31.419234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7d77-7ce3-4db0-8aaf-3afce69000dd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:31.423168 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:31.423803 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.424087 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.424378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.434940 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:31.436435 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:31.447645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1400/2754] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:31.554663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:31.556271 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:31.569585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983d6d1-e8d5-4568-86ec-0f062fdae5c7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:31.573199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d32389-e92e-4529-9a8f-e9b058eca8d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:31.573905 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32389-e92e-4529-9a8f-e9b058eca8d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:31.574183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32389-e92e-4529-9a8f-e9b058eca8d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:31.574450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32389-e92e-4529-9a8f-e9b058eca8d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:31.650795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d32389-e92e-4529-9a8f-e9b058eca8d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:31.652585 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d32389-e92e-4529-9a8f-e9b058eca8d6 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:31.664023 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1355/2755] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:32.008931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:32.076385 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580e658-b74b-4ba9-b300-d20cf1feb24a service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:32.080882 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:32.081750 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.082034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.082308 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] PATH_INFO: `/v3/users/d5818087c86e42edbd78eed19706bcda` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.084389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5818087c86e42edbd78eed19706bcda)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:32.086354 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:32.139600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5818087c86e42edbd78eed19706bcda'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.140389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5818087c86e42edbd78eed19706bcda'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.141091 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5818087c86e42edbd78eed19706bcda'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.148895 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5818087c86e42edbd78eed19706bcda'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.161402 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5818087c86e42edbd78eed19706bcda'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.175103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1401/2756] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:32 2022] DELETE /identity/v3/users/d5818087c86e42edbd78eed19706bcda => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:32.179785 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.180239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.180378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.197209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:32.198448 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.239393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:32.259436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1356/2757] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:32.269161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:32.271520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.329081 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72e8ea-0e4e-44c1-9251-54425deca0af admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:32.333330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:32.333962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.334243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.334520 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.347128 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:32.348644 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:32.358675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1402/2758] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:32.547520 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:32.548974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.560816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216771d-616e-4f8e-9e86-bc850e64d151 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:32.565348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:32.566212 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.566494 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.566768 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] PATH_INFO: `/v3/users/8feb891b1be946398e7c35cc34a8f150` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.568806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8feb891b1be946398e7c35cc34a8f150)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:32.570698 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:32.620820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8feb891b1be946398e7c35cc34a8f150'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.621597 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8feb891b1be946398e7c35cc34a8f150'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.630486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8feb891b1be946398e7c35cc34a8f150'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.631435 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8feb891b1be946398e7c35cc34a8f150'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.639540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2dd93185-77ed-4d93-85aa-f2f103823602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8feb891b1be946398e7c35cc34a8f150'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.641889 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:32.643731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.658787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a94c-4341-4e3f-85b4-d95b74d09bd0 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:32.663959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1357/2759] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:32 2022] DELETE /identity/v3/users/8feb891b1be946398e7c35cc34a8f150 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:32.668130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64351528-7e93-439d-9682-02469c41bad3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.668467 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64351528-7e93-439d-9682-02469c41bad3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.668752 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64351528-7e93-439d-9682-02469c41bad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.669277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:32.670488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.670873 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.671329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] PATH_INFO: `/v3/projects/e3d1df6ad615404d96ecd12eb7fe96ca` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.672872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d1df6ad615404d96ecd12eb7fe96ca)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:32.675050 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:32.677311 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:32.687975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-64351528-7e93-439d-9682-02469c41bad3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:32.689277 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-64351528-7e93-439d-9682-02469c41bad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.727963 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d1df6ad615404d96ecd12eb7fe96ca', 'request_id': 'req-db6c9a98-c2fb-4144-b68d-b41cab84a6da'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.734031 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d1df6ad615404d96ecd12eb7fe96ca', 'request_id': 'req-db6c9a98-c2fb-4144-b68d-b41cab84a6da'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.741027 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d1df6ad615404d96ecd12eb7fe96ca', 'request_id': 'req-db6c9a98-c2fb-4144-b68d-b41cab84a6da'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.750198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1358/2760] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:32.753604 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] The token cache is being invalidate because project e3d1df6ad615404d96ecd12eb7fe96ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:32.753893 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.755119 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.756722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1403/2761] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:32 2022] DELETE /identity/v3/projects/e3d1df6ad615404d96ecd12eb7fe96ca => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:32.760652 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:32.760814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64351528-7e93-439d-9682-02469c41bad3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:32.762064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.762187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-64351528-7e93-439d-9682-02469c41bad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.819513 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64351528-7e93-439d-9682-02469c41bad3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:32.827122 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c9a98-c2fb-4144-b68d-b41cab84a6da admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:32.828522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:32.832360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:32.833850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.834235 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.834491 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] PATH_INFO: `/v3/users/e207ef589b8744768eee9dc96918fedb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.835739 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e207ef589b8744768eee9dc96918fedb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:32.837897 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:32.874564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:32.875216 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:32.875496 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:32.875763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:32.878393 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:32.889590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e207ef589b8744768eee9dc96918fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.890790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e207ef589b8744768eee9dc96918fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.891495 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e207ef589b8744768eee9dc96918fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.892272 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:32.893742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:32.904283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1359/2762] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:32.907801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e207ef589b8744768eee9dc96918fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.923809 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e207ef589b8744768eee9dc96918fedb'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:32.940613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1404/2763] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:32 2022] DELETE /identity/v3/users/e207ef589b8744768eee9dc96918fedb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:33.277419 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.278866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:33.291115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ff0fe-0879-4370-a224-2982f9481290 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.307782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.309259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:33.325538 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:33.360873 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.361744 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.362027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.362301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] PATH_INFO: `/v3/projects/0b137cb155a44f5e942abe6bfaa53884` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.363545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b137cb155a44f5e942abe6bfaa53884)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.366198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.386176 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf05997-1539-49ef-a8f2-3a34f20228f7 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.391177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.392066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.392347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.392645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] PATH_INFO: `/v3/projects/4b028eda9a194c37b6eb3f0d4ad710a9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.394018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b028eda9a194c37b6eb3f0d4ad710a9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.396680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.416926 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b137cb155a44f5e942abe6bfaa53884', 'request_id': 'req-33887c7f-6b37-4334-a458-870d0a3a4665'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.423064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b137cb155a44f5e942abe6bfaa53884', 'request_id': 'req-33887c7f-6b37-4334-a458-870d0a3a4665'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.431942 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b137cb155a44f5e942abe6bfaa53884', 'request_id': 'req-33887c7f-6b37-4334-a458-870d0a3a4665'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.443276 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] The token cache is being invalidate because project 0b137cb155a44f5e942abe6bfaa53884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:33.443564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.444191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-33887c7f-6b37-4334-a458-870d0a3a4665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.445507 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b028eda9a194c37b6eb3f0d4ad710a9', 'request_id': 'req-c8b51232-ec81-4d09-8943-c5329901b227'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.446011 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1360/2764] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:33 2022] DELETE /identity/v3/projects/0b137cb155a44f5e942abe6bfaa53884 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:33.451442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b028eda9a194c37b6eb3f0d4ad710a9', 'request_id': 'req-c8b51232-ec81-4d09-8943-c5329901b227'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.459239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b028eda9a194c37b6eb3f0d4ad710a9', 'request_id': 'req-c8b51232-ec81-4d09-8943-c5329901b227'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.474206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] The token cache is being invalidate because project 4b028eda9a194c37b6eb3f0d4ad710a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:33.474554 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.475239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-c8b51232-ec81-4d09-8943-c5329901b227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:33.476980 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1405/2765] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:33 2022] DELETE /identity/v3/projects/4b028eda9a194c37b6eb3f0d4ad710a9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:33.482263 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549356-b06d-4ff5-9538-7f4d2717981a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.482559 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549356-b06d-4ff5-9538-7f4d2717981a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.482839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549356-b06d-4ff5-9538-7f4d2717981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.503355 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ef549356-b06d-4ff5-9538-7f4d2717981a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:33.505223 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef549356-b06d-4ff5-9538-7f4d2717981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:33.551840 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:33.572429 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1361/2766] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:33.581757 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585815c6-9d3c-4d2b-80d2-53ac2deece1d None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.582044 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585815c6-9d3c-4d2b-80d2-53ac2deece1d None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.582323 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585815c6-9d3c-4d2b-80d2-53ac2deece1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.601985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-585815c6-9d3c-4d2b-80d2-53ac2deece1d None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:33.603220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-585815c6-9d3c-4d2b-80d2-53ac2deece1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:33.611716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1406/2767] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:33.616864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549356-b06d-4ff5-9538-7f4d2717981a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.627137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549356-b06d-4ff5-9538-7f4d2717981a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.632656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.633335 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.633620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.633898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.634639 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:06:33.634951 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.636634 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.653152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1362/2768] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:06:33.654563 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585815c6-9d3c-4d2b-80d2-53ac2deece1d None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.664707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585815c6-9d3c-4d2b-80d2-53ac2deece1d None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.669716 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.670389 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.670673 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.670946 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.671693 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.673645 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.702245 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1407/2769] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:33.706469 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.717446 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1244-f6ab-4b16-8ed9-943955c33ef8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.773191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.773857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.774140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.774412 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.775099 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.777047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.796866 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1363/2770] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:33 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:33.800933 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.811761 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b44383-7981-4a9d-914d-53502ab3de9d admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.816479 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.817156 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.817439 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.817713 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.818429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.820006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.830134 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1408/2771] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:33.834248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.843934 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dded3-46f3-4447-a030-c6ae309761a2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.849246 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3960842-4042-453d-9ff6-7bd21aaca61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.850047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3960842-4042-453d-9ff6-7bd21aaca61c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.850366 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3960842-4042-453d-9ff6-7bd21aaca61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.850676 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3960842-4042-453d-9ff6-7bd21aaca61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.851441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3960842-4042-453d-9ff6-7bd21aaca61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.853085 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3960842-4042-453d-9ff6-7bd21aaca61c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.863136 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1364/2772] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:33.868029 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:33.878028 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d4aa-3596-4007-947e-fe3f7ce6e603 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:33.882792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:33.883666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.883951 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.884229 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] PATH_INFO: `/v3/projects/80cb2a920e2e464a9d039e7991ed2274/users/3195a75238774e03a29164e5f2a9ec47/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.898798 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cb2a920e2e464a9d039e7991ed2274, user_id=3195a75238774e03a29164e5f2a9ec47, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:33.901475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:33.916267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1409/2773] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:33 2022] PUT /identity/v3/projects/80cb2a920e2e464a9d039e7991ed2274/users/3195a75238774e03a29164e5f2a9ec47/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:33.927349 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9abfc-cb99-4915-882f-dc077a3bf0d5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:33.927773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9abfc-cb99-4915-882f-dc077a3bf0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:33.928144 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9abfc-cb99-4915-882f-dc077a3bf0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:33.955953 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f4a9abfc-cb99-4915-882f-dc077a3bf0d5 None None] MFA Rules not processed for user `3195a75238774e03a29164e5f2a9ec47`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:33.957187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f4a9abfc-cb99-4915-882f-dc077a3bf0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:34.022773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1365/2774] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:34.033579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:34.035276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:34.075306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:34.093388 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b14f40-b3ec-4b17-81d9-3ca36d7c69e9 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:34.097038 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:34.097676 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:34.098499 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:34.098752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:34.107221 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:34.108670 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:34.117949 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1410/2775] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:42.489965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d667-26ab-481b-84a0-90f078af1a38 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:42.490466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d667-26ab-481b-84a0-90f078af1a38 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:42.490568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d667-26ab-481b-84a0-90f078af1a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:42.507564 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8bf0d667-26ab-481b-84a0-90f078af1a38 None None] MFA Rules not processed for user `4244e60b4d424bf29575b8214aadde9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:42.508773 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8bf0d667-26ab-481b-84a0-90f078af1a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:42.545976 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:42.563281 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1366/2776] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:44.113860 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=10 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:47.715381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:47.797301 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a211278-266c-4245-824d-f1f1ac2aa23b service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:47.802725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:47.803612 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:47.803894 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:47.804169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] PATH_INFO: `/v3/users/3195a75238774e03a29164e5f2a9ec47` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:47.805407 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3195a75238774e03a29164e5f2a9ec47)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:47.807346 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:47.853580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3195a75238774e03a29164e5f2a9ec47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:47.854142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3195a75238774e03a29164e5f2a9ec47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:47.854737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3195a75238774e03a29164e5f2a9ec47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:47.861850 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3195a75238774e03a29164e5f2a9ec47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:47.874049 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3195a75238774e03a29164e5f2a9ec47'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:47.886766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1411/2777] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:47 2022] DELETE /identity/v3/users/3195a75238774e03a29164e5f2a9ec47 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:47.891336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:47.891849 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:47.891952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:47.909039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:47.910338 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:47.969197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1367/2778] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:47.978917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:47.980297 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:48.037347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c138c-b5e7-424f-af65-8df7e30cf11f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.040929 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52342320-7e00-4ded-b87e-6e57f2ff8ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.041569 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52342320-7e00-4ded-b87e-6e57f2ff8ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.041846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52342320-7e00-4ded-b87e-6e57f2ff8ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.042115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52342320-7e00-4ded-b87e-6e57f2ff8ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.052045 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52342320-7e00-4ded-b87e-6e57f2ff8ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.053474 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52342320-7e00-4ded-b87e-6e57f2ff8ed0 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.062726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1412/2779] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:48.325603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:48.327025 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:48.340249 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c810afa-c6c0-41aa-bdbf-dbff049f1537 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.345125 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.346262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.346262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.346536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] PATH_INFO: `/v3/projects/80cb2a920e2e464a9d039e7991ed2274` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.348305 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cb2a920e2e464a9d039e7991ed2274)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.350728 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.397742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cb2a920e2e464a9d039e7991ed2274', 'request_id': 'req-e16c2005-9500-465a-8790-32df6b14e1f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:48.403952 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cb2a920e2e464a9d039e7991ed2274', 'request_id': 'req-e16c2005-9500-465a-8790-32df6b14e1f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:48.411628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cb2a920e2e464a9d039e7991ed2274', 'request_id': 'req-e16c2005-9500-465a-8790-32df6b14e1f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:48.423114 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] The token cache is being invalidate because project 80cb2a920e2e464a9d039e7991ed2274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:48.423396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:48.424497 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e16c2005-9500-465a-8790-32df6b14e1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:48.425841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1368/2780] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:48 2022] DELETE /identity/v3/projects/80cb2a920e2e464a9d039e7991ed2274 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:48.430960 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaea2d-57a8-41ca-8772-6989a786c84f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.431272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaea2d-57a8-41ca-8772-6989a786c84f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.431544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaea2d-57a8-41ca-8772-6989a786c84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.449064 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-feaaea2d-57a8-41ca-8772-6989a786c84f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:48.450240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-feaaea2d-57a8-41ca-8772-6989a786c84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:48.468541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:48.520752 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1413/2781] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:48.528655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e7583-ae2e-43f1-8bb2-96e77da5a5c6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.528944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e7583-ae2e-43f1-8bb2-96e77da5a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.529222 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e7583-ae2e-43f1-8bb2-96e77da5a5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.549300 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c41e7583-ae2e-43f1-8bb2-96e77da5a5c6 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:48.550705 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c41e7583-ae2e-43f1-8bb2-96e77da5a5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:48.558530 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1369/2782] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:48.562784 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaea2d-57a8-41ca-8772-6989a786c84f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:48.572418 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaea2d-57a8-41ca-8772-6989a786c84f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.576950 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.577599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.577881 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.578151 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] PATH_INFO: `/v3/domains` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.578869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80319) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:06:48.579171 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.580796 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.593067 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1414/2783] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:06:48.597147 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e7583-ae2e-43f1-8bb2-96e77da5a5c6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:48.606867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e7583-ae2e-43f1-8bb2-96e77da5a5c6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.611914 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.612630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.612898 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.613215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.613998 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.615895 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.642005 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1370/2784] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:48 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:48.645454 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:48.655555 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a87ead-0ea6-4ee1-b153-b5f015a0b395 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.715557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.716427 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.716711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.716988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.717689 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.719651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.745665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1415/2785] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:48 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:48.746883 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:48.758152 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73b24f-203a-4ce7-9990-434906fef2cd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.763030 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.763699 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.763982 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.764260 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.765120 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.766680 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.780660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1371/2786] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:48.784707 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:48.797731 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8985dd4-49d4-448f-90f3-e891ab478207 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.805959 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cc4d5-1e09-42fe-a5d6-d68a9a70d968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.807026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cc4d5-1e09-42fe-a5d6-d68a9a70d968 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.807295 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cc4d5-1e09-42fe-a5d6-d68a9a70d968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.807619 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cc4d5-1e09-42fe-a5d6-d68a9a70d968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.808656 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cc4d5-1e09-42fe-a5d6-d68a9a70d968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.810760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cc4d5-1e09-42fe-a5d6-d68a9a70d968 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.825345 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1416/2787] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:48.829532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:48.850741 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e4c4-1c10-4433-84de-30bb9d71c7fc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:48.856681 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:48.857835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.858122 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.858401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] PATH_INFO: `/v3/projects/29430ef310694e1297af8e22f34fcf94/users/76db1414a93046028f80b9f4d1582eb4/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.878491 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29430ef310694e1297af8e22f34fcf94, user_id=76db1414a93046028f80b9f4d1582eb4, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:48.882193 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:48.902720 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1372/2788] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:48 2022] PUT /identity/v3/projects/29430ef310694e1297af8e22f34fcf94/users/76db1414a93046028f80b9f4d1582eb4/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:48.968233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0d04-6653-43ce-ac4e-0903527217c6 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:48.968544 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0d04-6653-43ce-ac4e-0903527217c6 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:48.968827 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0d04-6653-43ce-ac4e-0903527217c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:48.996452 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4e9d0d04-6653-43ce-ac4e-0903527217c6 None None] MFA Rules not processed for user `76db1414a93046028f80b9f4d1582eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:48.997660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4e9d0d04-6653-43ce-ac4e-0903527217c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:49.070872 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1417/2789] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:49.075342 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.085723 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:49.146640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cc5ff-d875-43cd-ac18-32c6f9bb0ba7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.151427 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.152121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.152381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.152684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.153395 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.155303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.182221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1373/2790] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:49 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:49.186101 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0d04-6653-43ce-ac4e-0903527217c6 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.197611 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0d04-6653-43ce-ac4e-0903527217c6 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.210021 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:49.389108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.389785 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.390062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.390335 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.391025 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.392966 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.413415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1418/2791] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:49 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:49.417488 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.428054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b559ac3-afe9-4f1c-830b-8feb9f108120 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.433237 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.433831 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.434111 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.434384 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.435101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.436756 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.447367 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1374/2792] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:49.451149 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.462017 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc3485-3eb5-4c83-92eb-9af03bbf5232 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.466728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d7147-ecb7-49ed-9c88-c08d0fc3dc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.467387 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7147-ecb7-49ed-9c88-c08d0fc3dc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.467698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7147-ecb7-49ed-9c88-c08d0fc3dc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.467971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7147-ecb7-49ed-9c88-c08d0fc3dc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.468758 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d7147-ecb7-49ed-9c88-c08d0fc3dc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.470338 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d7147-ecb7-49ed-9c88-c08d0fc3dc5f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.481121 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1419/2793] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:49.485543 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.495651 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fae03f-04f9-4559-9a80-68612a62a245 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.500184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.501093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.501378 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.501654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] PATH_INFO: `/v3/projects/0726ddf9cac241888599e00622e5da6d/users/635b70eb090844fea409093d5b1515c8/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.516666 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0726ddf9cac241888599e00622e5da6d, user_id=635b70eb090844fea409093d5b1515c8, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.519303 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.533476 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1375/2794] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:49 2022] PUT /identity/v3/projects/0726ddf9cac241888599e00622e5da6d/users/635b70eb090844fea409093d5b1515c8/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:49.544175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126eb30-22d0-436d-acf1-22dd0386496b None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.544500 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126eb30-22d0-436d-acf1-22dd0386496b None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.544775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126eb30-22d0-436d-acf1-22dd0386496b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.574859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-c126eb30-22d0-436d-acf1-22dd0386496b None None] MFA Rules not processed for user `635b70eb090844fea409093d5b1515c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:49.574859 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-c126eb30-22d0-436d-acf1-22dd0386496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:49.644941 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1420/2795] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:49.649985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.712916 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a0abd-fd47-444a-856f-073d73375f3d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.717985 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.718633 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.718913 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.719185 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.719857 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.721763 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.750538 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1376/2796] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:49 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:49.751632 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126eb30-22d0-436d-acf1-22dd0386496b None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.762401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126eb30-22d0-436d-acf1-22dd0386496b None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.819330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.820488 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.820992 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.821552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.822616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.824717 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.848802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1421/2797] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:49 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:49.853158 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.866298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5ab6f-e7bc-4ef9-b4e7-a8344a54a18b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.872283 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.873012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.873933 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.874236 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.874964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.877279 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.890194 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1377/2798] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:49.894906 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.904816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8bed-fa74-4b21-87e2-5c5afa6a37bc admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.909426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:49.910938 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:49.911408 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:49.911688 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] PATH_INFO: `/v3/projects/b99cd0b1382943d496cff9cbfab92df7/users/940d86d4bebc400e83a8173c728508b4/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:49.923557 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:49.927361 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99cd0b1382943d496cff9cbfab92df7, user_id=940d86d4bebc400e83a8173c728508b4, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:49.930622 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:49.943441 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1422/2799] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:49 2022] PUT /identity/v3/projects/b99cd0b1382943d496cff9cbfab92df7/users/940d86d4bebc400e83a8173c728508b4/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:49.947360 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:49.958423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913889d-ca91-4689-80dc-96fd9b5bdff8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:49.988296 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:50.016865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35806692-492f-4b5a-b4c3-54bdb3e4fc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:50.017532 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35806692-492f-4b5a-b4c3-54bdb3e4fc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:50.017816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35806692-492f-4b5a-b4c3-54bdb3e4fc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:50.018092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35806692-492f-4b5a-b4c3-54bdb3e4fc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:50.018835 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35806692-492f-4b5a-b4c3-54bdb3e4fc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:50.021040 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35806692-492f-4b5a-b4c3-54bdb3e4fc41 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:50.031007 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1378/2800] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:50.036059 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:50.046365 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc32f8-cee9-47c6-8bf4-4dd536b32ea1 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:50.051143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:50.052012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:50.052290 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:50.052602 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] PATH_INFO: `/v3/projects/b99cd0b1382943d496cff9cbfab92df7/users/940d86d4bebc400e83a8173c728508b4/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:50.054564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99cd0b1382943d496cff9cbfab92df7, user_id=940d86d4bebc400e83a8173c728508b4, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:50.057191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:50.067313 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1423/2801] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:50 2022] PUT /identity/v3/projects/b99cd0b1382943d496cff9cbfab92df7/users/940d86d4bebc400e83a8173c728508b4/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:50.078354 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:50.078730 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:50.079019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:50.106379 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] MFA Rules not processed for user `940d86d4bebc400e83a8173c728508b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:50.107821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:50.191656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1379/2802] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:50.203256 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:50.205012 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:50.271296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f13dbb-c355-4f39-9b4c-6cab91c2cdac admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:50.276905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a39c-75ec-439f-9107-8baea6909375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:50.277552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a39c-75ec-439f-9107-8baea6909375 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:50.277834 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a39c-75ec-439f-9107-8baea6909375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:50.278102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a39c-75ec-439f-9107-8baea6909375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:50.336299 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a39c-75ec-439f-9107-8baea6909375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:50.337934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a39c-75ec-439f-9107-8baea6909375 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:50.346861 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1424/2803] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:50.709519 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:50.710978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:50.775936 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c7b8a-d636-46c6-8db4-9a145d7239d3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:50.780645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:50.781516 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:50.781796 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:50.782070 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] PATH_INFO: `/v3/users/9d60eca9c7f040ba896aa628ec10dbbd` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:50.783313 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d60eca9c7f040ba896aa628ec10dbbd)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:50.785450 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:50.836487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d60eca9c7f040ba896aa628ec10dbbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:50.837128 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d60eca9c7f040ba896aa628ec10dbbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:50.842385 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:50.844301 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d60eca9c7f040ba896aa628ec10dbbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:50.845198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d60eca9c7f040ba896aa628ec10dbbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:50.853170 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d60eca9c7f040ba896aa628ec10dbbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:50.870814 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1380/2804] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:50 2022] DELETE /identity/v3/users/9d60eca9c7f040ba896aa628ec10dbbd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:50.875760 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:50.876324 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:50.876579 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:50.895981 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:50.897280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:50.957616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1425/2805] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:50.967612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:50.969958 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:51.032800 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98f6b4-2626-46bd-ae98-fb0a00d10436 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.037915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaca35a-4a6d-4300-866c-38a946211726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.038499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaca35a-4a6d-4300-866c-38a946211726 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.038779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaca35a-4a6d-4300-866c-38a946211726 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.039053 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaca35a-4a6d-4300-866c-38a946211726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.048417 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaca35a-4a6d-4300-866c-38a946211726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.049847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaca35a-4a6d-4300-866c-38a946211726 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.060700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1381/2806] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:51.332296 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.335074 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:51.351329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ffc9b-0cf2-41b0-9aec-19672ff26bf8 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.357671 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.358537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.358823 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.359096 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] PATH_INFO: `/v3/projects/8229792d46e54e28b79208db9b611451` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.361118 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8229792d46e54e28b79208db9b611451)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.364199 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.397975 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:51.417680 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8229792d46e54e28b79208db9b611451', 'request_id': 'req-78a079ac-44b1-406e-b6b5-47705756bc1b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:51.423246 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8229792d46e54e28b79208db9b611451', 'request_id': 'req-78a079ac-44b1-406e-b6b5-47705756bc1b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:51.430772 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8229792d46e54e28b79208db9b611451', 'request_id': 'req-78a079ac-44b1-406e-b6b5-47705756bc1b'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:51.445922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] The token cache is being invalidate because project 8229792d46e54e28b79208db9b611451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:51.446343 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:51.447429 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78a079ac-44b1-406e-b6b5-47705756bc1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:51.452666 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1426/2807] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:51 2022] DELETE /identity/v3/projects/8229792d46e54e28b79208db9b611451 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:51.454545 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a4db-0d8c-43dd-b600-66c010a80a10 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.455252 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a4db-0d8c-43dd-b600-66c010a80a10 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.455697 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a4db-0d8c-43dd-b600-66c010a80a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.487031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7ba0a4db-0d8c-43dd-b600-66c010a80a10 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:51.488323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7ba0a4db-0d8c-43dd-b600-66c010a80a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:51.551047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1382/2808] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:51.559321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91742a-9314-48ba-8602-d4dd6c58df4f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.559321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91742a-9314-48ba-8602-d4dd6c58df4f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.559590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91742a-9314-48ba-8602-d4dd6c58df4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.578179 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-4a91742a-9314-48ba-8602-d4dd6c58df4f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:51.579378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-4a91742a-9314-48ba-8602-d4dd6c58df4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:51.587672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1427/2809] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:51.591826 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a4db-0d8c-43dd-b600-66c010a80a10 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.602383 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a4db-0d8c-43dd-b600-66c010a80a10 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.606917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.607824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.608103 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.608406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.609180 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:06:51.609486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.611082 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.623081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1383/2810] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:06:51.627321 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91742a-9314-48ba-8602-d4dd6c58df4f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.637472 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91742a-9314-48ba-8602-d4dd6c58df4f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.642254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.642727 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.643006 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.643276 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.643956 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.646175 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.674932 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1428/2811] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:51 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:51.679656 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.689630 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231ebb-b305-4eb5-b184-8162517b873e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.750994 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.751663 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.751972 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.752239 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.752973 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.755215 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.762966 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:51.780993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1384/2812] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:51 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:06:51.782173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.793166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12072a-0502-433a-9654-4bc235b4c55f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.798401 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.799051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.799330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.799709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.800378 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.802625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.815934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1429/2813] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:51.817499 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.830137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7aebf2-e979-46d2-8aba-d20eea6ce72c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.835625 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.836404 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.836550 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.836695 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.836970 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.837694 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.838186 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:51.839278 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.853583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1385/2814] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:51.854580 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:51.865112 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397cc1b-24c3-4a84-8949-cd4ed591f51a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.869669 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1237e-e5d7-45c1-909d-ede414181629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.870556 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1237e-e5d7-45c1-909d-ede414181629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.870838 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1237e-e5d7-45c1-909d-ede414181629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.871113 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1237e-e5d7-45c1-909d-ede414181629 admin admin] PATH_INFO: `/v3/projects/0598199897de4461b994edd96e987dfb/users/bce573cefd1641559d57ba0b0d2a8b3c/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.886409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1237e-e5d7-45c1-909d-ede414181629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0598199897de4461b994edd96e987dfb, user_id=bce573cefd1641559d57ba0b0d2a8b3c, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:51.889093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1237e-e5d7-45c1-909d-ede414181629 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:51.902590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4751-e8c9-4fec-82f9-0d74f5128763 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:51.904855 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1386/2815] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:51 2022] PUT /identity/v3/projects/0598199897de4461b994edd96e987dfb/users/bce573cefd1641559d57ba0b0d2a8b3c/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:51.909606 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.909911 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.910920 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.928659 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:51.929963 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:51.956054 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:51.956718 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:51.957094 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:51.957360 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:51.959133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:51.991839 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1387/2816] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:52.001370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:52.002886 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:52.008392 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:52.024253 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:52.025718 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:52.034665 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1430/2817] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:52.061507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa885f-aec8-4299-b4b5-fa986224e5ce None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:52.065079 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:52.065738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:52.066019 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:52.066915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:52.076380 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:52.077819 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:52.088154 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1388/2818] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:06:52.630763 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:52.639603 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111f9b9-df92-4516-926f-a733bf097497 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:52.643594 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a801bc-5fa3-4143-9ca3-305764cfcb98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:52.644233 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a801bc-5fa3-4143-9ca3-305764cfcb98 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:52.645202 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a801bc-5fa3-4143-9ca3-305764cfcb98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:52.645660 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a801bc-5fa3-4143-9ca3-305764cfcb98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:52.647728 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a5a801bc-5fa3-4143-9ca3-305764cfcb98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:52.716205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a801bc-5fa3-4143-9ca3-305764cfcb98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:52.717847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a801bc-5fa3-4143-9ca3-305764cfcb98 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:52.728833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1431/2819] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:56.787514 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:56.789078 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:56.802064 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96ea0b-e587-4d52-b760-4b40f8b6fe82 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:56.808140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:56.808955 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:56.809229 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:56.809967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] PATH_INFO: `/v3/users/76db1414a93046028f80b9f4d1582eb4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:56.811209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76db1414a93046028f80b9f4d1582eb4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:56.813187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:56.903667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76db1414a93046028f80b9f4d1582eb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:56.904602 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76db1414a93046028f80b9f4d1582eb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:56.911054 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:56.913421 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76db1414a93046028f80b9f4d1582eb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:56.914411 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76db1414a93046028f80b9f4d1582eb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:56.931574 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76db1414a93046028f80b9f4d1582eb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:56.960115 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1389/2820] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:56 2022] DELETE /identity/v3/users/76db1414a93046028f80b9f4d1582eb4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:56.964724 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee960-93b6-4578-af5f-ed69eeb2208a None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:56.965272 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee960-93b6-4578-af5f-ed69eeb2208a None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:56.965382 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee960-93b6-4578-af5f-ed69eeb2208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:56.989123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-262ee960-93b6-4578-af5f-ed69eeb2208a None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:56.990352 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-262ee960-93b6-4578-af5f-ed69eeb2208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:57.077815 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1432/2821] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:06:57.090221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.092903 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:57.183441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31729b5d-38a5-4d92-a553-f7b14f192802 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.187654 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.188724 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.189265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.189628 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.200557 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.203012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.216856 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1390/2822] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:06:57.263173 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18024676-e9c7-4099-8748-86386ccc97a0 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.263493 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18024676-e9c7-4099-8748-86386ccc97a0 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.264137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18024676-e9c7-4099-8748-86386ccc97a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.296917 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-18024676-e9c7-4099-8748-86386ccc97a0 None None] MFA Rules not processed for user `bce573cefd1641559d57ba0b0d2a8b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:06:57.298145 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-18024676-e9c7-4099-8748-86386ccc97a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:57.385155 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1433/2823] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:06:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:06:57.390588 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.392490 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:57.417637 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df493b72-2ee0-4420-85cf-81d43fa07333 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.422645 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.423388 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.423743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.424092 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.424917 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.427370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.457409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1391/2824] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:06:57 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:57.461450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18024676-e9c7-4099-8748-86386ccc97a0 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.476478 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18024676-e9c7-4099-8748-86386ccc97a0 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.489265 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:57.559837 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.573157 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.573599 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.573912 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] PATH_INFO: `/v3/users` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.576257 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.578377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.596621 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.599162 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:57.611795 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1434/2825] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:06:57 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:06:57.612636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.623552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ff744-6ed6-4f98-b930-6abdcb5b6deb admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.627101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25e11b-a060-40aa-a659-fedbf7e1b73e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.629018 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.630086 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.630377 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.631687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.631687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.632973 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.633299 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.634165 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.634451 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.634729 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] PATH_INFO: `/v3/projects/29430ef310694e1297af8e22f34fcf94` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.636231 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29430ef310694e1297af8e22f34fcf94)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.638197 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.651430 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1435/2826] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:57.652597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.667381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c85fc-82d8-4577-8877-148fbc5bcc71 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.674306 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.675228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.675541 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.675916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] PATH_INFO: `/v3/projects/73040879400849428417033cff2e77b0/users/c40ec5bbe7fb4b41b28cb5fc940a5045/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.700927 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=73040879400849428417033cff2e77b0, user_id=c40ec5bbe7fb4b41b28cb5fc940a5045, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.703792 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.723089 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1436/2827] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:57 2022] PUT /identity/v3/projects/73040879400849428417033cff2e77b0/users/c40ec5bbe7fb4b41b28cb5fc940a5045/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:57.725824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29430ef310694e1297af8e22f34fcf94', 'request_id': 'req-deb83195-1672-4418-acdc-ceed5d33a063'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.727659 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.737400 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29430ef310694e1297af8e22f34fcf94', 'request_id': 'req-deb83195-1672-4418-acdc-ceed5d33a063'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.752865 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29430ef310694e1297af8e22f34fcf94', 'request_id': 'req-deb83195-1672-4418-acdc-ceed5d33a063'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.786782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] The token cache is being invalidate because project 29430ef310694e1297af8e22f34fcf94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:57.787129 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.787807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.793931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1392/2828] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:57 2022] DELETE /identity/v3/projects/29430ef310694e1297af8e22f34fcf94 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:57.795084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.797433 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:57.846254 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d192017-57ac-44ba-b540-61c087535b1f admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.853979 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.854999 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.855279 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.855597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.856331 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.859275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.865052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:57.882033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1437/2829] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:06:57.886124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:57.888288 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:57.900732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb83195-1672-4418-acdc-ceed5d33a063 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.903283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3e58a-a0ae-462b-aa47-c2cee5c50a20 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:57.907748 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.907934 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:57.908671 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.908790 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:57.908961 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.909217 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] PATH_INFO: `/v3/users/635b70eb090844fea409093d5b1515c8` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.909613 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:57.910464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] PATH_INFO: `/v3/projects/73040879400849428417033cff2e77b0/users/c40ec5bbe7fb4b41b28cb5fc940a5045/roles/5b74a32ffd254d2fbe8013955251c61e` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:57.911340 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] RBAC: Authorizing `identity:delete_user(user_id=635b70eb090844fea409093d5b1515c8)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.913640 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.919347 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73040879400849428417033cff2e77b0, user_id=c40ec5bbe7fb4b41b28cb5fc940a5045, role_id=5b74a32ffd254d2fbe8013955251c61e)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:57.921995 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:57.937523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1438/2830] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:06:57 2022] PUT /identity/v3/projects/73040879400849428417033cff2e77b0/users/c40ec5bbe7fb4b41b28cb5fc940a5045/roles/5b74a32ffd254d2fbe8013955251c61e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:06:57.970950 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '635b70eb090844fea409093d5b1515c8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.971679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '635b70eb090844fea409093d5b1515c8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.982015 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '635b70eb090844fea409093d5b1515c8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.983262 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '635b70eb090844fea409093d5b1515c8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:57.993833 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '635b70eb090844fea409093d5b1515c8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:58.011827 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1393/2831] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:57 2022] DELETE /identity/v3/users/635b70eb090844fea409093d5b1515c8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:58.563364 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:58.565103 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:58.647669 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:58.667725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8170-e91d-4222-879a-d45fd093b9c8 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:58.757232 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:58.759403 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:58.760033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:58.760278 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] PATH_INFO: `/v3/projects/0726ddf9cac241888599e00622e5da6d` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:58.763066 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0726ddf9cac241888599e00622e5da6d)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:58.765682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:58.857722 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0726ddf9cac241888599e00622e5da6d', 'request_id': 'req-f26e2e07-2030-417c-acf8-2ef22119c8a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:58.869616 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0726ddf9cac241888599e00622e5da6d', 'request_id': 'req-f26e2e07-2030-417c-acf8-2ef22119c8a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:58.879808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0726ddf9cac241888599e00622e5da6d', 'request_id': 'req-f26e2e07-2030-417c-acf8-2ef22119c8a0'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:58.898007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] The token cache is being invalidate because project 0726ddf9cac241888599e00622e5da6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:06:58.899115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:58.899788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:58.902322 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1439/2832] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:58 2022] DELETE /identity/v3/projects/0726ddf9cac241888599e00622e5da6d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:58.906121 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:58.920977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1bd72-619d-4780-9e42-aeedf419f46e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:06:59.089413 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:06:59.090382 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:06:59.090660 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:06:59.091273 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] PATH_INFO: `/v3/users/940d86d4bebc400e83a8173c728508b4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:06:59.092774 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=940d86d4bebc400e83a8173c728508b4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:06:59.097002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:06:59.171370 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '940d86d4bebc400e83a8173c728508b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:59.172155 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '940d86d4bebc400e83a8173c728508b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:59.180573 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '940d86d4bebc400e83a8173c728508b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:59.181646 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '940d86d4bebc400e83a8173c728508b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:59.183670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:06:59.195667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-52b6d187-575c-4371-9d33-ac448ab917a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '940d86d4bebc400e83a8173c728508b4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:06:59.237721 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1394/2833] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:06:58 2022] DELETE /identity/v3/users/940d86d4bebc400e83a8173c728508b4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:06:59.804106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:06:59.807095 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:06:59.860677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2e07-2030-417c-acf8-2ef22119c8a0 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:00.024677 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:00.025564 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.025841 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.026112 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] PATH_INFO: `/v3/projects/b99cd0b1382943d496cff9cbfab92df7` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.027400 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99cd0b1382943d496cff9cbfab92df7)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:00.029409 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:00.088961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:00.114532 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99cd0b1382943d496cff9cbfab92df7', 'request_id': 'req-9f5ec373-3ed1-4f86-8372-c2424b346a02'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:00.123190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99cd0b1382943d496cff9cbfab92df7', 'request_id': 'req-9f5ec373-3ed1-4f86-8372-c2424b346a02'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:00.149083 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99cd0b1382943d496cff9cbfab92df7', 'request_id': 'req-9f5ec373-3ed1-4f86-8372-c2424b346a02'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:00.188922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] The token cache is being invalidate because project b99cd0b1382943d496cff9cbfab92df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:07:00.189386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:00.191853 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9f5ec373-3ed1-4f86-8372-c2424b346a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:00.196259 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1440/2834] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:06:59 2022] DELETE /identity/v3/projects/b99cd0b1382943d496cff9cbfab92df7 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:07:00.202750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3dc46-572f-4632-b188-6d6b7a246fda None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.203211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3dc46-572f-4632-b188-6d6b7a246fda None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.203211 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3dc46-572f-4632-b188-6d6b7a246fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.247909 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c4c3dc46-572f-4632-b188-6d6b7a246fda None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:00.249438 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c4c3dc46-572f-4632-b188-6d6b7a246fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:00.344501 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1395/2835] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:00.352918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009008da-41a2-4edb-a1d5-25e10244228f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.353255 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009008da-41a2-4edb-a1d5-25e10244228f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.353475 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009008da-41a2-4edb-a1d5-25e10244228f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.386228 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-009008da-41a2-4edb-a1d5-25e10244228f None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:00.387708 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-009008da-41a2-4edb-a1d5-25e10244228f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:00.403988 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1441/2836] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:00.405474 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3dc46-572f-4632-b188-6d6b7a246fda None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:00.421544 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3dc46-572f-4632-b188-6d6b7a246fda None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:00.432441 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:00.433507 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.433964 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.434682 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.436323 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:07:00.436809 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:00.439266 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:00.457613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1396/2837] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 16:07:00.461668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009008da-41a2-4edb-a1d5-25e10244228f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:00.475786 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009008da-41a2-4edb-a1d5-25e10244228f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:00.482793 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:00.483903 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.484374 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.484849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.485922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:00.489001 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:00.532381 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1442/2838] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:07:00 2022] POST /identity/v3/projects => generated 397 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 16:07:00.536410 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:00.548851 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631ae4-87a5-4b35-a966-d0909f04b3e9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:00.698268 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:00.698974 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.699258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.699536 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.700251 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:00.702221 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:00.744727 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1397/2839] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:07:00 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:07:00.747945 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:00.766240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea4356-59b1-4a9d-b9a5-0296339289dd admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:00.812783 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:00.814460 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.815426 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.828503 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.829775 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:00.833187 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:00.848105 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1443/2840] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:07:00.851841 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:00.878081 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec8da1-0e8a-4916-8ba5-7d35503ec38e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:00.887062 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302294-08ee-405c-88c8-0ff68339a062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:00.899965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302294-08ee-405c-88c8-0ff68339a062 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.900331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302294-08ee-405c-88c8-0ff68339a062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.900711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302294-08ee-405c-88c8-0ff68339a062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.901581 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302294-08ee-405c-88c8-0ff68339a062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:00.903426 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302294-08ee-405c-88c8-0ff68339a062 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:00.919949 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1398/2841] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:07:00.920636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:00.933982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f94947-b646-4e4f-b502-87e384ac8b1e admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:00.949062 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:00.950125 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:00.950411 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:00.950682 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] PATH_INFO: `/v3/projects/fec7ae04a59545f5b16e78f6460e6f51/users/5ba8a35b904d488986a4b8c776279a0d/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:00.981294 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec7ae04a59545f5b16e78f6460e6f51, user_id=5ba8a35b904d488986a4b8c776279a0d, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:00.984985 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:01.021849 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1444/2842] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:07:00 2022] PUT /identity/v3/projects/fec7ae04a59545f5b16e78f6460e6f51/users/5ba8a35b904d488986a4b8c776279a0d/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 16:07:01.031929 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc6c0a-759b-42d4-a1cc-80505c78014d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:01.032297 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc6c0a-759b-42d4-a1cc-80505c78014d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:01.032693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc6c0a-759b-42d4-a1cc-80505c78014d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:01.089210 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2cbc6c0a-759b-42d4-a1cc-80505c78014d None None] MFA Rules not processed for user `5ba8a35b904d488986a4b8c776279a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:01.090977 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2cbc6c0a-759b-42d4-a1cc-80505c78014d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:01.173571 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:01.272436 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1399/2843] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:01.283811 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:01.287647 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:01.420720 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d440e12-b8d2-4c08-b210-d080467902a5 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:01.433060 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:01.433726 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:01.434008 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:01.434273 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:01.453697 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:01.459170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:01.483314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1445/2844] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:07:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:07:03.427304 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396020c-3e0c-4a49-a5dd-5ed7d72aee0d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:03.427813 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396020c-3e0c-4a49-a5dd-5ed7d72aee0d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:03.428258 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396020c-3e0c-4a49-a5dd-5ed7d72aee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:03.457031 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a396020c-3e0c-4a49-a5dd-5ed7d72aee0d None None] MFA Rules not processed for user `c40ec5bbe7fb4b41b28cb5fc940a5045`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:03.458130 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a396020c-3e0c-4a49-a5dd-5ed7d72aee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:03.535584 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1400/2845] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:03.552709 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:03.554315 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:03.618161 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3bb07-6646-4bf9-9f56-70ecfbb6f256 service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:03.621751 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac0206e09e4f426d9f4b54543d89c039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:03.622394 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:03.622672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:03.622943 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:03.624740 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:03.635609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:03.692180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:03.698848 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:03.704600 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1446/2846] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:07:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:07:03.815946 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d72d1-1706-4c76-9579-4b204a5ba5d7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:03.816242 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d72d1-1706-4c76-9579-4b204a5ba5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:03.816578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d72d1-1706-4c76-9579-4b204a5ba5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:03.834529 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-598d72d1-1706-4c76-9579-4b204a5ba5d7 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:03.835799 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-598d72d1-1706-4c76-9579-4b204a5ba5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:03.917623 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1401/2847] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:07:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:03.967537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:03.968971 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:04.047034 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c68d6f-9c83-4728-bf86-a7d00b56b3f7 service glance] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:04.052237 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:04.052922 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:04.053205 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:04.053477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:04.062595 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:04.064097 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:04.073725 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1447/2848] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:07:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:07:05.718816 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50fb83-d0c4-4889-a184-2e8b042cd5ce None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:05.719353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50fb83-d0c4-4889-a184-2e8b042cd5ce None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:05.719353 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50fb83-d0c4-4889-a184-2e8b042cd5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:05.754737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0e50fb83-d0c4-4889-a184-2e8b042cd5ce None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:05.756648 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0e50fb83-d0c4-4889-a184-2e8b042cd5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:05.770470 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1402/2849] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:07:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:05.786327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:05.801471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cb2ca-f57f-4606-b72f-9a4447cca335 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:05.830282 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb9929-9fce-4b3f-902a-c18a046dbbba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:05.831007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb9929-9fce-4b3f-902a-c18a046dbbba service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:05.831399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb9929-9fce-4b3f-902a-c18a046dbbba service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:05.831735 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb9929-9fce-4b3f-902a-c18a046dbbba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:05.847854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb9929-9fce-4b3f-902a-c18a046dbbba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:05.849833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb9929-9fce-4b3f-902a-c18a046dbbba service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:05.867240 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1448/2850] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:07:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:07:12.594336 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50fb83-d0c4-4889-a184-2e8b042cd5ce None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:12.707311 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:12.730096 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50fb83-d0c4-4889-a184-2e8b042cd5ce None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:12.735148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:12.736093 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:12.736484 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:12.736864 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] PATH_INFO: `/v3/users/5ba8a35b904d488986a4b8c776279a0d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:12.738610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba8a35b904d488986a4b8c776279a0d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:12.740641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:12.822265 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba8a35b904d488986a4b8c776279a0d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:12.822935 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba8a35b904d488986a4b8c776279a0d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:12.837465 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba8a35b904d488986a4b8c776279a0d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:12.839098 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba8a35b904d488986a4b8c776279a0d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:12.856477 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba8a35b904d488986a4b8c776279a0d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:12.915675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1403/2851] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:07:12 2022] DELETE /identity/v3/users/5ba8a35b904d488986a4b8c776279a0d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:07:12.920180 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:12.920744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:12.920744 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:12.953341 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:12.954910 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:13.046106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1449/2852] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:13.046915 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:13.049069 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:13.164486 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcb554-68ad-45d6-87e5-8d8dd8c5fc55 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:13.169655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f72cfb-a92d-4987-b612-84d0b0379103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:13.170324 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f72cfb-a92d-4987-b612-84d0b0379103 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:13.170604 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f72cfb-a92d-4987-b612-84d0b0379103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:13.170896 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f72cfb-a92d-4987-b612-84d0b0379103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:13.186003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f72cfb-a92d-4987-b612-84d0b0379103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:13.187679 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f72cfb-a92d-4987-b612-84d0b0379103 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:13.222959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1404/2853] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:07:13.632222 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:13.634414 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:13.662691 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cbc78-715b-45e7-a3e5-5f7bca61acb3 None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:13.683618 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:13.684534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:13.684816 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:13.685088 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] PATH_INFO: `/v3/projects/fec7ae04a59545f5b16e78f6460e6f51` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:13.686329 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec7ae04a59545f5b16e78f6460e6f51)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:13.690120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:13.739919 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=100 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:13.794239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec7ae04a59545f5b16e78f6460e6f51', 'request_id': 'req-b3238822-9b14-4837-9176-924a8756a9be'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:13.804552 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec7ae04a59545f5b16e78f6460e6f51', 'request_id': 'req-b3238822-9b14-4837-9176-924a8756a9be'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:13.811524 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec7ae04a59545f5b16e78f6460e6f51', 'request_id': 'req-b3238822-9b14-4837-9176-924a8756a9be'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:13.826730 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] The token cache is being invalidate because project fec7ae04a59545f5b16e78f6460e6f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:07:13.827078 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:13.827747 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:13.841133 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1450/2854] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:07:13 2022] DELETE /identity/v3/projects/fec7ae04a59545f5b16e78f6460e6f51 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:07:16.037610 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32969534-dd50-4386-8c8b-d7a2ab2729f4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:16.038084 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32969534-dd50-4386-8c8b-d7a2ab2729f4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:16.038423 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32969534-dd50-4386-8c8b-d7a2ab2729f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:16.057108 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-32969534-dd50-4386-8c8b-d7a2ab2729f4 None None] MFA Rules not processed for user `bce573cefd1641559d57ba0b0d2a8b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:16.058593 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-32969534-dd50-4386-8c8b-d7a2ab2729f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:16.121881 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1405/2855] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:17.668771 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:17.670325 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:17.764120 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3238822-9b14-4837-9176-924a8756a9be admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:17.771051 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:17.771808 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:17.772166 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:17.772539 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:17.774835 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:17.793283 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:17.795174 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:17.807446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1451/2856] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:07:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:07:18.915160 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fe2a9-23d0-47a8-9320-6a7acd47e502 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:18.915847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fe2a9-23d0-47a8-9320-6a7acd47e502 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:18.915847 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fe2a9-23d0-47a8-9320-6a7acd47e502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:18.937390 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d40fe2a9-23d0-47a8-9320-6a7acd47e502 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:18.938641 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d40fe2a9-23d0-47a8-9320-6a7acd47e502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:18.955401 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:19.037508 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1406/2857] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Jun 7 16:07:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:19.046547 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:19.048597 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:19.150847 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d936-f600-4636-b2a3-7dc5495a4868 service placement] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:19.158227 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:19.159172 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:19.159609 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:19.159948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:19.169019 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:19.170766 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:19.185450 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1452/2858] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:07:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:07:24.699620 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:24.700199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:24.700199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:24.718810 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:24.720016 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:24.728655 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1407/2859] 10.222.0.18 () {58 vars in 967 bytes} [Tue Jun 7 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:24.742026 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:24.754190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9d1d-8433-4fa0-a7c9-e5128381b35e service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:24.759636 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:24.760277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:24.760706 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:24.760984 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:24.770188 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:24.771782 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:24.788334 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1453/2860] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:07:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:07:27.176126 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:27.177607 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:27.254600 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533d8a-3a7e-458c-ab49-0162cb924965 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:27.259569 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:27.260472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:27.260757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:27.261037 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] PATH_INFO: `/v3/users/4244e60b4d424bf29575b8214aadde9a` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:27.262460 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4244e60b4d424bf29575b8214aadde9a)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:27.264424 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:27.332140 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4244e60b4d424bf29575b8214aadde9a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.333090 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4244e60b4d424bf29575b8214aadde9a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.342461 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4244e60b4d424bf29575b8214aadde9a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.343330 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4244e60b4d424bf29575b8214aadde9a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.356389 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b62b782e-bf07-4574-8ff4-102770c24ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4244e60b4d424bf29575b8214aadde9a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.411825 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1408/2861] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:07:27 2022] DELETE /identity/v3/users/4244e60b4d424bf29575b8214aadde9a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:07:27.651327 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:27.671061 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239ab5f-85d4-4212-af36-3213b4eb44e4 service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:27.686292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:27.740410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:27.740410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:27.740410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:27.740410 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] PATH_INFO: `/v3/projects/aad305bb9275482b95b858fa3669e8e9` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:27.740957 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad305bb9275482b95b858fa3669e8e9)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:27.742249 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:27.795152 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad305bb9275482b95b858fa3669e8e9', 'request_id': 'req-d4203702-62b6-46b3-8689-0916914978fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.801633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad305bb9275482b95b858fa3669e8e9', 'request_id': 'req-d4203702-62b6-46b3-8689-0916914978fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.808092 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad305bb9275482b95b858fa3669e8e9', 'request_id': 'req-d4203702-62b6-46b3-8689-0916914978fc'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.821340 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] The token cache is being invalidate because project aad305bb9275482b95b858fa3669e8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:07:27.821626 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.822239 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-d4203702-62b6-46b3-8689-0916914978fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:07:27.823788 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1454/2862] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:07:27 2022] DELETE /identity/v3/projects/aad305bb9275482b95b858fa3669e8e9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:07:27.828927 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e6a28-54cf-4763-8017-024cbff28fc5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:27.829220 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e6a28-54cf-4763-8017-024cbff28fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:27.829504 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e6a28-54cf-4763-8017-024cbff28fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:27.847700 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a44e6a28-54cf-4763-8017-024cbff28fc5 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:27.848925 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a44e6a28-54cf-4763-8017-024cbff28fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:27.882298 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:27.910243 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1409/2863] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:27.917913 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962a05b-8bcc-4957-bf4d-254ed9fba1fd None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:27.918206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962a05b-8bcc-4957-bf4d-254ed9fba1fd None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:27.918485 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962a05b-8bcc-4957-bf4d-254ed9fba1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:27.936039 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-d962a05b-8bcc-4957-bf4d-254ed9fba1fd None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:27.937243 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-d962a05b-8bcc-4957-bf4d-254ed9fba1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:27.945509 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1455/2864] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 16:07:27.949563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e6a28-54cf-4763-8017-024cbff28fc5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:27.960039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e6a28-54cf-4763-8017-024cbff28fc5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:27.964707 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:27.965381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:27.965667 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:27.965944 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:27.966689 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 16:07:27.967039 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:27.968743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:27.984867 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1410/2865] 10.222.0.18 () {58 vars in 1152 bytes} [Tue Jun 7 16:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 16:07:27.987463 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962a05b-8bcc-4957-bf4d-254ed9fba1fd None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:27.996719 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962a05b-8bcc-4957-bf4d-254ed9fba1fd None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:28.001280 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:28.001916 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:28.002194 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:28.002464 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] PATH_INFO: `/v3/projects` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:28.003939 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:28.005845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:28.030711 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1456/2866] 10.222.0.18 () {60 vars in 1152 bytes} [Tue Jun 7 16:07:27 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:07:28.035255 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:28.046391 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd39d57-d41d-451a-bbec-a295c92a69f1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:28.102795 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:28.103466 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:28.103754 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:28.104035 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:28.104757 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:28.106675 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:28.125830 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1411/2867] 10.222.0.18 () {60 vars in 1143 bytes} [Tue Jun 7 16:07:28 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 16:07:28.130921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:28.141330 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff73aa9-78e1-4f17-833c-3cd4161e866c admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:28.145876 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:28.146523 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:28.146801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:28.147072 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:28.147801 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:28.149397 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:28.159802 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1457/2868] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 16:07:28.163820 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:28.173622 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f75d-c2b1-42eb-9724-d48d0763170b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:28.178782 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb80d-9293-4169-8c9e-ff69d1143876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:28.179409 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb80d-9293-4169-8c9e-ff69d1143876 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:28.179693 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb80d-9293-4169-8c9e-ff69d1143876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:28.179965 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb80d-9293-4169-8c9e-ff69d1143876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:28.180726 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fb80d-9293-4169-8c9e-ff69d1143876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:28.182291 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fb80d-9293-4169-8c9e-ff69d1143876 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:28.192735 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1412/2869] 10.222.0.18 () {58 vars in 1121 bytes} [Tue Jun 7 16:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 16:07:28.196986 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:28.206994 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b064a7-0bc8-43a3-858c-7a226130c222 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:28.211787 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:28.212741 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:28.213146 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:28.213530 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] PATH_INFO: `/v3/projects/ab43bc41a9b44512b7e3e4c1f721bacf/users/8725c1db5ee3468d881a42a3f2df50ea/roles/98ebd7033f624d02bef09757068ee356` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:28.228234 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab43bc41a9b44512b7e3e4c1f721bacf, user_id=8725c1db5ee3468d881a42a3f2df50ea, role_id=98ebd7033f624d02bef09757068ee356)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:28.231007 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:28.244169 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1458/2870] 10.222.0.18 () {60 vars in 1482 bytes} [Tue Jun 7 16:07:28 2022] PUT /identity/v3/projects/ab43bc41a9b44512b7e3e4c1f721bacf/users/8725c1db5ee3468d881a42a3f2df50ea/roles/98ebd7033f624d02bef09757068ee356 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:07:28.249148 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444908c3-5e30-4428-8b9c-381991a4abc2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:28.249440 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444908c3-5e30-4428-8b9c-381991a4abc2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:28.249717 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444908c3-5e30-4428-8b9c-381991a4abc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:28.267287 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-444908c3-5e30-4428-8b9c-381991a4abc2 None None] MFA Rules not processed for user `bd49b97115e3460492249a2e84a91b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:28.269225 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-444908c3-5e30-4428-8b9c-381991a4abc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:28.327089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1413/2871] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:28.337292 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:28.338982 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:28.384833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:28.398962 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e4a2-c7ed-41f4-a4d5-1288a5a7ae14 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:28.402578 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:28.403206 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:28.403481 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:28.403749 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:28.413826 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:28.415230 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:28.424667 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1459/2872] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 16:07:29.057259 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8d53d-e03e-4d50-8262-3b9b4b538525 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:29.057551 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8d53d-e03e-4d50-8262-3b9b4b538525 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:29.057829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8d53d-e03e-4d50-8262-3b9b4b538525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:29.086608 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-44b8d53d-e03e-4d50-8262-3b9b4b538525 None None] MFA Rules not processed for user `8725c1db5ee3468d881a42a3f2df50ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:29.087747 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-44b8d53d-e03e-4d50-8262-3b9b4b538525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:29.124746 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:29.165605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1414/2873] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:29.176715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:29.178148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:29.237862 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10159676-683a-4598-ae06-82685ea99ccb service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:29.241473 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a31ce1720fa84c43aec2ea016371f475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:29.242137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:29.242415 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:29.242712 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:29.250972 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:29.252921 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:29.261562 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1460/2874] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:07:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:07:30.617546 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa25cf-c0fb-4266-9e2e-ae80cb7dfd28 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:30.618066 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa25cf-c0fb-4266-9e2e-ae80cb7dfd28 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:30.618178 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa25cf-c0fb-4266-9e2e-ae80cb7dfd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:30.635346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c1fa25cf-c0fb-4266-9e2e-ae80cb7dfd28 None None] MFA Rules not processed for user `8725c1db5ee3468d881a42a3f2df50ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:07:30.636563 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c1fa25cf-c0fb-4266-9e2e-ae80cb7dfd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:30.643995 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1415/2875] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:07:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:07:39.257675 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=33 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:40.630101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=15 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:07:48.984184 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:07:48.986163 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:48.998351 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f7495-ce01-4790-8c62-2e27b8d553ef service cinder] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:07:49.002032 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:07:49.002672 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:07:49.002953 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:07:49.003226 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:07:49.005013 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:07:49.089518 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:07:49.090948 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:07:49.100339 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1461/2876] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:07:59.096124 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=30 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:08:16.841876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa25cf-c0fb-4266-9e2e-ae80cb7dfd28 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:08:16.851612 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa25cf-c0fb-4266-9e2e-ae80cb7dfd28 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:08:16.855742 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:08:16.856453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:16.856737 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:16.857012 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:16.859042 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:16.871821 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:08:16.873397 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:08:16.883954 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1416/2877] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:08:22.968104 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:08:22.969576 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:22.985203 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a0b5-e65c-4144-aadf-b47f5912bf1c service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:08:22.990442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:08:22.991314 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:22.991596 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:22.991869 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] PATH_INFO: `/v3/users/c26cd29ec472479a9a36ac9bb1f05580` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:22.993856 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26cd29ec472479a9a36ac9bb1f05580)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:08:22.995748 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:08:23.041574 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26cd29ec472479a9a36ac9bb1f05580'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.042211 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26cd29ec472479a9a36ac9bb1f05580'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.042833 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26cd29ec472479a9a36ac9bb1f05580'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.049573 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26cd29ec472479a9a36ac9bb1f05580'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.062108 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26cd29ec472479a9a36ac9bb1f05580'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.075137 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1462/2878] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:08:22 2022] DELETE /identity/v3/users/c26cd29ec472479a9a36ac9bb1f05580 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:08:23.245315 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:08:23.256000 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3c5b7-b7fb-4715-8e71-5654fa49d0ed service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:08:23.321568 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:08:23.322422 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:23.322702 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:23.322975 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] PATH_INFO: `/v3/projects/44a1ab295b3a423aa2b3183a3e25ec0b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:23.324137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a1ab295b3a423aa2b3183a3e25ec0b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:08:23.326102 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:08:23.372047 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a1ab295b3a423aa2b3183a3e25ec0b', 'request_id': 'req-5ff12afe-4229-4e20-ad71-36caa344796e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.378664 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a1ab295b3a423aa2b3183a3e25ec0b', 'request_id': 'req-5ff12afe-4229-4e20-ad71-36caa344796e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.385941 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a1ab295b3a423aa2b3183a3e25ec0b', 'request_id': 'req-5ff12afe-4229-4e20-ad71-36caa344796e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.396187 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] The token cache is being invalidate because project 44a1ab295b3a423aa2b3183a3e25ec0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:08:23.396512 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.397137 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.399184 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1417/2879] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:08:23 2022] DELETE /identity/v3/projects/44a1ab295b3a423aa2b3183a3e25ec0b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:08:23.403106 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:08:23.404537 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:23.418170 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30709f0e-8887-4216-8198-10971b980492 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:08:23.478162 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:08:23.480040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:23.480040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:23.480040 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] PATH_INFO: `/v3/users/4fc158cd498a450a82248cba8a3a3eac` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:23.481277 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc158cd498a450a82248cba8a3a3eac)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:08:23.483492 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:08:23.532905 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc158cd498a450a82248cba8a3a3eac'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.533705 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc158cd498a450a82248cba8a3a3eac'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.534558 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc158cd498a450a82248cba8a3a3eac'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.545487 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc158cd498a450a82248cba8a3a3eac'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.561614 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-78dc423e-20e2-404d-ad73-b70bfc165fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc158cd498a450a82248cba8a3a3eac'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.563736 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:08:23.580715 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1463/2880] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:08:23 2022] DELETE /identity/v3/users/4fc158cd498a450a82248cba8a3a3eac => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:08:23.787670 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:08:23.789386 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:23.802603 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff12afe-4229-4e20-ad71-36caa344796e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:08:23.861626 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:08:23.862492 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:23.862779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:23.863057 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] PATH_INFO: `/v3/projects/34304975d4e94e8eabf8794ccd98e2fd` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:23.864247 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34304975d4e94e8eabf8794ccd98e2fd)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:08:23.866179 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:08:23.888547 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:08:23.913806 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34304975d4e94e8eabf8794ccd98e2fd', 'request_id': 'req-b332097a-c850-44bd-b241-7e146b65c378'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.920017 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34304975d4e94e8eabf8794ccd98e2fd', 'request_id': 'req-b332097a-c850-44bd-b241-7e146b65c378'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.927371 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34304975d4e94e8eabf8794ccd98e2fd', 'request_id': 'req-b332097a-c850-44bd-b241-7e146b65c378'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.938332 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] The token cache is being invalidate because project 34304975d4e94e8eabf8794ccd98e2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:08:23.938618 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.939240 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:08:23.940804 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1418/2881] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:08:23 2022] DELETE /identity/v3/projects/34304975d4e94e8eabf8794ccd98e2fd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:08:33.579477 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:08:33.935425 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:08:36.116625 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c7991-c601-4649-881f-5131a5b0319f None None] REQUEST_METHOD: `POST` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:36.117177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c7991-c601-4649-881f-5131a5b0319f None None] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:36.117177 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c7991-c601-4649-881f-5131a5b0319f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:36.134534 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.auth.core [None req-721c7991-c601-4649-881f-5131a5b0319f None None] MFA Rules not processed for user `8725c1db5ee3468d881a42a3f2df50ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80319) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:08:36.135874 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-721c7991-c601-4649-881f-5131a5b0319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:36.191269 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1464/2882] 10.222.0.18 () {58 vars in 957 bytes} [Tue Jun 7 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:08:44.458684 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:08:44.460110 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:44.517172 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332097a-c850-44bd-b241-7e146b65c378 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:08:44.520767 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:08:44.521406 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:44.521686 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:44.521959 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:44.523738 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:44.593999 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:08:44.595432 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:08:44.606578 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1419/2883] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:08:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:08:46.184591 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=25 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:08:46.926383 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c7991-c601-4649-881f-5131a5b0319f None None] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:08:46.936918 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c7991-c601-4649-881f-5131a5b0319f None None] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:08:46.940466 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:08:46.941102 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:08:46.941386 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:08:46.941668 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:08:46.943438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:08:46.956845 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:08:46.958293 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:08:46.969746 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1465/2884] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:08:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:08:54.601445 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=54 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:08:56.964887 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=6 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:09:32.304190 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:32.305710 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:32.371668 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd292d-ed97-4691-b676-3060c67f8242 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:32.376153 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618981a0-abba-4c80-9203-10840937308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:32.377715 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618981a0-abba-4c80-9203-10840937308b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:32.377993 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618981a0-abba-4c80-9203-10840937308b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:32.378452 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618981a0-abba-4c80-9203-10840937308b admin admin] PATH_INFO: `/v3/users/8725c1db5ee3468d881a42a3f2df50ea` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:32.379649 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618981a0-abba-4c80-9203-10840937308b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8725c1db5ee3468d881a42a3f2df50ea)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:32.381605 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618981a0-abba-4c80-9203-10840937308b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:32.427403 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8725c1db5ee3468d881a42a3f2df50ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.428002 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8725c1db5ee3468d881a42a3f2df50ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.435072 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8725c1db5ee3468d881a42a3f2df50ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.435711 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8725c1db5ee3468d881a42a3f2df50ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.445094 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8725c1db5ee3468d881a42a3f2df50ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.466517 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1420/2885] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:09:32 2022] DELETE /identity/v3/users/8725c1db5ee3468d881a42a3f2df50ea => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:09:32.638657 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:32.649390 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ee4f7-ed79-4662-ac86-5eb9a44ca3f5 service neutron] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:32.709799 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:32.710651 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:32.710925 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:32.711190 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] PATH_INFO: `/v3/projects/ab43bc41a9b44512b7e3e4c1f721bacf` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:32.712337 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab43bc41a9b44512b7e3e4c1f721bacf)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:32.714220 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:32.760289 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab43bc41a9b44512b7e3e4c1f721bacf', 'request_id': 'req-8fb8662b-be0d-43aa-905d-ecd08ca955c4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.765931 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab43bc41a9b44512b7e3e4c1f721bacf', 'request_id': 'req-8fb8662b-be0d-43aa-905d-ecd08ca955c4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.772123 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab43bc41a9b44512b7e3e4c1f721bacf', 'request_id': 'req-8fb8662b-be0d-43aa-905d-ecd08ca955c4'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.787698 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] The token cache is being invalidate because project ab43bc41a9b44512b7e3e4c1f721bacf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:09:32.788143 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.789115 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:32.790633 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1466/2886] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:09:32 2022] DELETE /identity/v3/projects/ab43bc41a9b44512b7e3e4c1f721bacf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:09:34.672224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:34.673824 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:34.748870 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618981a0-abba-4c80-9203-10840937308b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:34.791583 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:09:34.826807 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ffb7-6549-437a-b69f-3ab198979917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dba9f1fa257a4f44ae159850f9ba349c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:34.827487 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ffb7-6549-437a-b69f-3ab198979917 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:34.827829 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ffb7-6549-437a-b69f-3ab198979917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:34.828209 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ffb7-6549-437a-b69f-3ab198979917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:34.830052 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e0c6ffb7-6549-437a-b69f-3ab198979917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:34.843150 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6ffb7-6549-437a-b69f-3ab198979917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:34.844732 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6ffb7-6549-437a-b69f-3ab198979917 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:34.854475 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1421/2887] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:09:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:09:41.461840 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:41.463471 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:41.543846 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8662b-be0d-43aa-905d-ecd08ca955c4 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:41.549627 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b352e6a51fc4d1f8b9b97d092d11bd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:41.551399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:41.551399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:41.551399 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:41.553209 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:41.566033 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:41.567590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:41.578372 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1467/2888] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:09:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:09:42.366101 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:42.366561 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:42.366665 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:42.383992 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] MFA Rules not processed for user `d95f13b6e27c43a7868a8ef79621eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 16:09:42.385198 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:42.439978 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1422/2889] 10.222.0.18 () {58 vars in 967 bytes} [Tue Jun 7 16:09:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 16:09:42.485446 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:42.486839 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:42.505737 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:09:42.557854 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8699d-2cc3-44a8-8f6d-5e15c508a1c1 service nova] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:42.562806 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31137a8197ab4072b47646b893a9733a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:42.563442 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] REQUEST_METHOD: `GET` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:42.563723 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:42.563998 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:42.572131 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:42.573753 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:42.581936 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1468/2890] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Jun 7 16:09:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 16:09:48.874248 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:48.876004 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:48.953275 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67402fb-ffcc-4249-a497-6df9bb30c880 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:48.958142 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:48.959020 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:48.959346 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:48.959619 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] PATH_INFO: `/v3/users/bce573cefd1641559d57ba0b0d2a8b3c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:48.960844 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce573cefd1641559d57ba0b0d2a8b3c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:48.962784 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:49.007579 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce573cefd1641559d57ba0b0d2a8b3c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.008224 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce573cefd1641559d57ba0b0d2a8b3c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.015089 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce573cefd1641559d57ba0b0d2a8b3c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.015718 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce573cefd1641559d57ba0b0d2a8b3c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.023743 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce573cefd1641559d57ba0b0d2a8b3c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.042205 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1423/2891] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:09:48 2022] DELETE /identity/v3/users/bce573cefd1641559d57ba0b0d2a8b3c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:09:49.209878 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:49.220214 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a88de-dc51-489a-95f8-eb31effac92a service swift] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:49.277734 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:49.278590 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:49.278870 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:49.279142 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] PATH_INFO: `/v3/projects/0598199897de4461b994edd96e987dfb` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:49.280307 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0598199897de4461b994edd96e987dfb)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:49.282267 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:49.327421 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0598199897de4461b994edd96e987dfb', 'request_id': 'req-5271e517-8e0e-4e10-9c33-8f5a36617b81'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.334212 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0598199897de4461b994edd96e987dfb', 'request_id': 'req-5271e517-8e0e-4e10-9c33-8f5a36617b81'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.339836 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0598199897de4461b994edd96e987dfb', 'request_id': 'req-5271e517-8e0e-4e10-9c33-8f5a36617b81'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.353298 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] The token cache is being invalidate because project 0598199897de4461b994edd96e987dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:09:49.353584 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.354191 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.355754 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1469/2892] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:09:49 2022] DELETE /identity/v3/projects/0598199897de4461b994edd96e987dfb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:09:49.360028 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:49.361453 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:49.369540 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:09:49.376199 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179774a4-bdeb-4d34-9dff-a3f222dd3d65 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:49.434314 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:49.435191 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:49.435472 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:49.435750 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] PATH_INFO: `/v3/users/c40ec5bbe7fb4b41b28cb5fc940a5045` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:49.436957 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c40ec5bbe7fb4b41b28cb5fc940a5045)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:49.438876 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:49.484381 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c40ec5bbe7fb4b41b28cb5fc940a5045'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.485348 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c40ec5bbe7fb4b41b28cb5fc940a5045'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.492331 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c40ec5bbe7fb4b41b28cb5fc940a5045'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.493003 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c40ec5bbe7fb4b41b28cb5fc940a5045'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.500183 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c40ec5bbe7fb4b41b28cb5fc940a5045'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.517396 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1424/2893] 10.222.0.18 () {58 vars in 1223 bytes} [Tue Jun 7 16:09:49 2022] DELETE /identity/v3/users/c40ec5bbe7fb4b41b28cb5fc940a5045 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:09:49.698438 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Authenticating user token {{(pid=80319) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:09:49.700252 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.fernet_utils [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80319) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:09:49.719687 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271e517-8e0e-4e10-9c33-8f5a36617b81 admin admin] Validating token access rules against request {{(pid=80319) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:09:49.757328 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:09:49.786148 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd49b97115e3460492249a2e84a91b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e30cc2a9bc04d579b04c2060b966f87', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80319) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:09:49.787016 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:09:49.787303 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:09:49.787580 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] PATH_INFO: `/v3/projects/73040879400849428417033cff2e77b0` {{(pid=80319) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:09:49.788824 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=73040879400849428417033cff2e77b0)` {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:09:49.790714 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] RBAC: Authorization granted {{(pid=80319) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:09:49.838020 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73040879400849428417033cff2e77b0', 'request_id': 'req-9575dc39-af51-47fc-a7cc-a40bc7891e9d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.843540 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73040879400849428417033cff2e77b0', 'request_id': 'req-9575dc39-af51-47fc-a7cc-a40bc7891e9d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.850393 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73040879400849428417033cff2e77b0', 'request_id': 'req-9575dc39-af51-47fc-a7cc-a40bc7891e9d'} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.862961 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] The token cache is being invalidate because project 73040879400849428417033cff2e77b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80319) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 16:09:49.863275 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.863967 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG keystone.notifications [None req-9575dc39-af51-47fc-a7cc-a40bc7891e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80319) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 16:09:49.865631 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: [pid: 80319|app: 0|req: 1470/2894] 10.222.0.18 () {58 vars in 1232 bytes} [Tue Jun 7 16:09:49 2022] DELETE /identity/v3/projects/73040879400849428417033cff2e77b0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 16:09:59.513815 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:09:59.859291 dv-cinder-844879-2-iscsi devstack@keystone.service[80319]: DEBUG dbcounter [-] [80319] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 16:12:57.546522 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 16:12:57.547931 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:12:57.688967 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83f048-19e4-4308-aba6-9e3307ee5156 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 16:12:57.749613 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab98588-49bc-40ac-808b-7c1d8298d290 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0372966e33f6413ebe529c001bbb2518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29a328fa91894ba5b320490897d0559b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 16:12:57.750181 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab98588-49bc-40ac-808b-7c1d8298d290 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 16:12:57.750485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab98588-49bc-40ac-808b-7c1d8298d290 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 16:12:57.750761 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab98588-49bc-40ac-808b-7c1d8298d290 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 16:12:57.752570 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8ab98588-49bc-40ac-808b-7c1d8298d290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 16:12:57.763894 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab98588-49bc-40ac-808b-7c1d8298d290 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 16:12:57.765485 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab98588-49bc-40ac-808b-7c1d8298d290 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 16:12:57.775779 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1425/2895] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Jun 7 16:12:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 16:13:07.771027 dv-cinder-844879-2-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}